From 8057576bb12e075ed3b3fd5d4363b9b52d4db088 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 18:21:48 +0200 Subject: [PATCH 01/17] Bump MSRV to 1.36 --- .travis.yml | 4 ++-- README.md | 14 ++++++-------- rand_chacha/README.md | 2 +- rand_core/README.md | 2 +- rand_distr/README.md | 2 +- rand_hc/README.md | 2 +- rand_pcg/README.md | 2 +- 7 files changed, 13 insertions(+), 15 deletions(-) diff --git a/.travis.yml b/.travis.yml index 06c25eac7fc..e0167f4c0fc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,8 +6,8 @@ sudo: false matrix: include: - - rust: 1.32.0 - name: "Linux, 1.32.0" + - rust: 1.36.0 + name: "Linux, 1.36.0" env: ALLOC=0 os: linux diff --git a/README.md b/README.md index bd7ec768a5a..0fc38546c99 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ [![Book](https://img.shields.io/badge/book-master-yellow.svg)](https://rust-random.github.io/book/) [![API](https://img.shields.io/badge/api-master-yellow.svg)](https://rust-random.github.io/rand) [![API](https://docs.rs/rand/badge.svg)](https://docs.rs/rand) -[![Minimum rustc version](https://img.shields.io/badge/rustc-1.32+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) +[![Minimum rustc version](https://img.shields.io/badge/rustc-1.36+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) A Rust library for random number generation. @@ -75,11 +75,11 @@ issue tracker with the keyword `yank` *should* uncover the motivation. ### Rust version requirements -Since version 0.7, Rand requires **Rustc version 1.32 or greater**. -Rand 0.5 requires Rustc 1.22 or greater while versions -0.4 and 0.3 (since approx. June 2017) require Rustc version 1.15 or -greater. Subsets of the Rand code may work with older Rust versions, but this -is not supported. +Since version 0.8, Rand requires **Rustc version 1.36 or greater**. +Rand 0.7 requires Rustc 1.32 or greater while versions 0.5 require Rustc 1.22 or +greater, and 0.4 and 0.3 (since approx. June 2017) require Rustc version 1.15 or +greater. Subsets of the Rand code may work with older Rust versions, but this is +not supported. Travis CI always has a build with a pinned version of Rustc matching the oldest supported Rust release. The current policy is that this can be updated in any @@ -91,8 +91,6 @@ Rand is built with these features enabled by default: - `std` enables functionality dependent on the `std` lib - `alloc` (implied by `std`) enables functionality requiring an allocator - (when using this feature in `no_std`, Rand requires Rustc version 1.36 or - greater) - `getrandom` (implied by `std`) is an optional dependency providing the code behind `rngs::OsRng` - `std_rng` enables inclusion of `StdRng`, `thread_rng` and `random` diff --git a/rand_chacha/README.md b/rand_chacha/README.md index 69a0ce7cf2a..b1928fd9cc8 100644 --- a/rand_chacha/README.md +++ b/rand_chacha/README.md @@ -6,7 +6,7 @@ [![Book](https://img.shields.io/badge/book-master-yellow.svg)](https://rust-random.github.io/book/) [![API](https://img.shields.io/badge/api-master-yellow.svg)](https://rust-random.github.io/rand/rand_chacha) [![API](https://docs.rs/rand_chacha/badge.svg)](https://docs.rs/rand_chacha) -[![Minimum rustc version](https://img.shields.io/badge/rustc-1.32+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) +[![Minimum rustc version](https://img.shields.io/badge/rustc-1.36+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) A cryptographically secure random number generator that uses the ChaCha algorithm. diff --git a/rand_core/README.md b/rand_core/README.md index 467e66f98b6..78c8083fc82 100644 --- a/rand_core/README.md +++ b/rand_core/README.md @@ -6,7 +6,7 @@ [![Book](https://img.shields.io/badge/book-master-yellow.svg)](https://rust-random.github.io/book/) [![API](https://img.shields.io/badge/api-master-yellow.svg)](https://rust-random.github.io/rand/rand_core) [![API](https://docs.rs/rand_core/badge.svg)](https://docs.rs/rand_core) -[![Minimum rustc version](https://img.shields.io/badge/rustc-1.32+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) +[![Minimum rustc version](https://img.shields.io/badge/rustc-1.36+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) Core traits and error types of the [rand] library, plus tools for implementing RNGs. diff --git a/rand_distr/README.md b/rand_distr/README.md index 68acd2f0799..9a67e5b3f93 100644 --- a/rand_distr/README.md +++ b/rand_distr/README.md @@ -6,7 +6,7 @@ [[![Book](https://img.shields.io/badge/book-master-yellow.svg)](https://rust-random.github.io/book/) [![API](https://img.shields.io/badge/api-master-yellow.svg)](https://rust-random.github.io/rand/rand_distr) [![API](https://docs.rs/rand_distr/badge.svg)](https://docs.rs/rand_distr) -[![Minimum rustc version](https://img.shields.io/badge/rustc-1.32+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) +[![Minimum rustc version](https://img.shields.io/badge/rustc-1.36+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) Implements a full suite of random number distributions sampling routines. diff --git a/rand_hc/README.md b/rand_hc/README.md index 36449c0c826..4d992fec0c5 100644 --- a/rand_hc/README.md +++ b/rand_hc/README.md @@ -6,7 +6,7 @@ [[![Book](https://img.shields.io/badge/book-master-yellow.svg)](https://rust-random.github.io/book/) [![API](https://img.shields.io/badge/api-master-yellow.svg)](https://rust-random.github.io/rand/rand_hc) [![API](https://docs.rs/rand_hc/badge.svg)](https://docs.rs/rand_hc) -[![Minimum rustc version](https://img.shields.io/badge/rustc-1.32+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) +[![Minimum rustc version](https://img.shields.io/badge/rustc-1.36+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) A cryptographically secure random number generator that uses the HC-128 algorithm. diff --git a/rand_pcg/README.md b/rand_pcg/README.md index 5b9205002ec..97204213c8d 100644 --- a/rand_pcg/README.md +++ b/rand_pcg/README.md @@ -6,7 +6,7 @@ [![Book](https://img.shields.io/badge/book-master-yellow.svg)](https://rust-random.github.io/book/) [![API](https://img.shields.io/badge/api-master-yellow.svg)](https://rust-random.github.io/rand/rand_pcg) [![API](https://docs.rs/rand_pcg/badge.svg)](https://docs.rs/rand_pcg) -[![Minimum rustc version](https://img.shields.io/badge/rustc-1.32+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) +[![Minimum rustc version](https://img.shields.io/badge/rustc-1.36+-lightgray.svg)](https://github.com/rust-random/rand#rust-version-requirements) Implements a selection of PCG random number generators. From eef82c4cd2776247fea3c243706c3eeb37f6d654 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 18:22:31 +0200 Subject: [PATCH 02/17] Reenable alloc tests for MSRV --- .travis.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index e0167f4c0fc..35d755e3527 100644 --- a/.travis.yml +++ b/.travis.yml @@ -8,7 +8,6 @@ matrix: include: - rust: 1.36.0 name: "Linux, 1.36.0" - env: ALLOC=0 os: linux - rust: stable From 0432b06d2c265fc5aba98a967a5a08fa5acfad11 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 20:32:49 +0200 Subject: [PATCH 03/17] Drop some unsafe code from `rand_core` The necessary standard library functions were stabilized with Rust 1.34. Our MSRV is 1.36. --- rand_core/src/impls.rs | 45 +++++++++--------------------------------- rand_core/src/le.rs | 28 +++++++------------------- 2 files changed, 16 insertions(+), 57 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 532555a3465..6cadcda69df 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -19,9 +19,6 @@ use crate::RngCore; use core::cmp::min; -use core::mem::size_of; -use core::ptr::copy_nonoverlapping; -use core::slice; /// Implement `next_u64` via `next_u32`, little-endian order. pub fn next_u64_via_u32(rng: &mut R) -> u64 { @@ -55,41 +52,13 @@ pub fn fill_bytes_via_next(rng: &mut R, dest: &mut [u8]) { } } -macro_rules! impl_uint_from_fill { - ($rng:expr, $ty:ty, $N:expr) => {{ - debug_assert!($N == size_of::<$ty>()); - - let mut int: $ty = 0; - unsafe { - let ptr = &mut int as *mut $ty as *mut u8; - let slice = slice::from_raw_parts_mut(ptr, $N); - $rng.fill_bytes(slice); - } - int - }}; -} - macro_rules! fill_via_chunks { ($src:expr, $dst:expr, $ty:ty, $size:expr) => {{ let chunk_size_u8 = min($src.len() * $size, $dst.len()); let chunk_size = (chunk_size_u8 + $size - 1) / $size; - if cfg!(target_endian = "little") { - unsafe { - copy_nonoverlapping( - $src.as_ptr() as *const u8, - $dst.as_mut_ptr(), - chunk_size_u8); - } - } else { - for (&n, chunk) in $src.iter().zip($dst.chunks_mut($size)) { - let tmp = n.to_le(); - let src_ptr = &tmp as *const $ty as *const u8; - unsafe { - copy_nonoverlapping(src_ptr, - chunk.as_mut_ptr(), - chunk.len()); - } - } + + for (&n, chunk) in $src.iter().zip($dst.chunks_mut($size)) { + chunk.clone_from_slice(&n.to_le_bytes()); } (chunk_size, chunk_size_u8) @@ -146,12 +115,16 @@ pub fn fill_via_u64_chunks(src: &[u64], dest: &mut [u8]) -> (usize, usize) { /// Implement `next_u32` via `fill_bytes`, little-endian order. pub fn next_u32_via_fill(rng: &mut R) -> u32 { - impl_uint_from_fill!(rng, u32, 4) + let mut buf = [0; 4]; + rng.fill_bytes(&mut buf); + u32::from_le_bytes(buf) } /// Implement `next_u64` via `fill_bytes`, little-endian order. pub fn next_u64_via_fill(rng: &mut R) -> u64 { - impl_uint_from_fill!(rng, u64, 8) + let mut buf = [0; 8]; + rng.fill_bytes(&mut buf); + u64::from_le_bytes(buf) } // TODO: implement tests for the above diff --git a/rand_core/src/le.rs b/rand_core/src/le.rs index c247ab7b3ff..69759be019d 100644 --- a/rand_core/src/le.rs +++ b/rand_core/src/le.rs @@ -11,36 +11,22 @@ //! Little-Endian order has been chosen for internal usage; this makes some //! useful functions available. -use core::ptr; - -macro_rules! read_slice { - ($src:expr, $dst:expr, $size:expr, $which:ident) => {{ - assert_eq!($src.len(), $size * $dst.len()); - - unsafe { - ptr::copy_nonoverlapping( - $src.as_ptr(), - $dst.as_mut_ptr() as *mut u8, - $src.len()); - } - for v in $dst.iter_mut() { - *v = v.$which(); - } - }}; -} +use core::convert::TryInto; /// Reads unsigned 32 bit integers from `src` into `dst`. -/// Borrowed from the `byteorder` crate. #[inline] pub fn read_u32_into(src: &[u8], dst: &mut [u32]) { - read_slice!(src, dst, 4, to_le); + for i in 0..dst.len() { + dst[i] = u32::from_le_bytes((&src[i..i + 4]).try_into().unwrap()); + } } /// Reads unsigned 64 bit integers from `src` into `dst`. -/// Borrowed from the `byteorder` crate. #[inline] pub fn read_u64_into(src: &[u8], dst: &mut [u64]) { - read_slice!(src, dst, 8, to_le); + for i in 0..dst.len() { + dst[i] = u64::from_le_bytes((&src[i..i + 8]).try_into().unwrap()); + } } #[test] From afc6bc6d89a259cd149429e2a3c5a6641ee0a61c Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 21:12:13 +0200 Subject: [PATCH 04/17] Avoid indexing --- rand_core/src/le.rs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/rand_core/src/le.rs b/rand_core/src/le.rs index 69759be019d..d7bf57d7c66 100644 --- a/rand_core/src/le.rs +++ b/rand_core/src/le.rs @@ -16,16 +16,16 @@ use core::convert::TryInto; /// Reads unsigned 32 bit integers from `src` into `dst`. #[inline] pub fn read_u32_into(src: &[u8], dst: &mut [u32]) { - for i in 0..dst.len() { - dst[i] = u32::from_le_bytes((&src[i..i + 4]).try_into().unwrap()); + for (out, chunk) in dst.iter_mut().zip(src.chunks(4)) { + *out = u32::from_le_bytes(chunk.try_into().unwrap()); } } /// Reads unsigned 64 bit integers from `src` into `dst`. #[inline] pub fn read_u64_into(src: &[u8], dst: &mut [u64]) { - for i in 0..dst.len() { - dst[i] = u64::from_le_bytes((&src[i..i + 8]).try_into().unwrap()); + for (out, chunk) in dst.iter_mut().zip(src.chunks(8)) { + *out = u64::from_le_bytes(chunk.try_into().unwrap()); } } From f4746e14efcd92f57ed53d70281f4bfb7504bafd Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 22:34:31 +0200 Subject: [PATCH 05/17] Simplify `cfg` logic This is possible thanks to `alloc` being implied by `std` builds since Rust 1.36. --- rand_core/src/error.rs | 2 ++ rand_core/src/lib.rs | 7 ++++--- rand_distr/src/lib.rs | 5 +---- src/distributions/mod.rs | 1 + src/distributions/other.rs | 2 +- src/distributions/weighted.rs | 2 +- src/distributions/weighted_index.rs | 2 +- src/lib.rs | 5 +++-- src/rng.rs | 2 +- src/rngs/adapter/read.rs | 2 ++ src/rngs/thread.rs | 1 + src/seq/index.rs | 9 +++------ src/seq/mod.rs | 2 +- 13 files changed, 22 insertions(+), 20 deletions(-) diff --git a/rand_core/src/error.rs b/rand_core/src/error.rs index e10840269ed..b11e170c02f 100644 --- a/rand_core/src/error.rs +++ b/rand_core/src/error.rs @@ -11,6 +11,8 @@ use core::fmt; use core::num::NonZeroU32; +#[cfg(feature = "std")] use std::boxed::Box; + /// Error type of random number generators /// /// In order to be compatible with `std` and `no_std`, this type has two diff --git a/rand_core/src/lib.rs b/rand_core/src/lib.rs index 464cb0e5937..a338f994088 100644 --- a/rand_core/src/lib.rs +++ b/rand_core/src/lib.rs @@ -36,14 +36,15 @@ #![deny(missing_debug_implementations)] #![doc(test(attr(allow(unused_variables), deny(warnings))))] #![allow(clippy::unreadable_literal)] -#![cfg_attr(not(feature = "std"), no_std)] #![cfg_attr(doc_cfg, feature(doc_cfg))] +#![no_std] use core::convert::AsMut; use core::default::Default; -#[cfg(all(feature = "alloc", not(feature = "std")))] extern crate alloc; -#[cfg(all(feature = "alloc", not(feature = "std")))] use alloc::boxed::Box; +#[cfg(feature = "std")] extern crate std; +#[cfg(feature = "alloc")] extern crate alloc; +#[cfg(feature = "alloc")] use alloc::boxed::Box; pub use error::Error; #[cfg(feature = "getrandom")] pub use os::OsRng; diff --git a/rand_distr/src/lib.rs b/rand_distr/src/lib.rs index 59a00b4e949..b7c10c5884d 100644 --- a/rand_distr/src/lib.rs +++ b/rand_distr/src/lib.rs @@ -72,14 +72,11 @@ //! - [`InverseGaussian`] distribution //! - [`NormalInverseGaussian`] distribution -#[cfg(all(feature = "alloc", not(feature = "std")))] +#[cfg(feature = "alloc")] extern crate alloc; #[cfg(feature = "std")] extern crate std; -// TODO: remove on MSRV bump to 1.36 -#[cfg(feature = "std")] -extern crate std as alloc; pub use rand::distributions::{ uniform, Alphanumeric, Bernoulli, BernoulliError, DistIter, Distribution, Open01, OpenClosed01, diff --git a/src/distributions/mod.rs b/src/distributions/mod.rs index 75c13a066a3..5b1f61d9cd6 100644 --- a/src/distributions/mod.rs +++ b/src/distributions/mod.rs @@ -330,6 +330,7 @@ pub struct Standard; #[cfg(all(test, feature = "std"))] mod tests { + use std::{vec::Vec, println}; use super::{Distribution, Uniform}; use crate::Rng; diff --git a/src/distributions/other.rs b/src/distributions/other.rs index b07ae9f1be6..f62fe59aa73 100644 --- a/src/distributions/other.rs +++ b/src/distributions/other.rs @@ -206,7 +206,7 @@ where Standard: Distribution mod tests { use super::*; use crate::RngCore; - #[cfg(all(not(feature = "std"), feature = "alloc"))] use alloc::string::String; + #[cfg(feature = "alloc")] use alloc::string::String; #[test] fn test_misc() { diff --git a/src/distributions/weighted.rs b/src/distributions/weighted.rs index 20248f51375..6dd9273a506 100644 --- a/src/distributions/weighted.rs +++ b/src/distributions/weighted.rs @@ -19,7 +19,7 @@ pub mod alias_method { // This module exists to provide a deprecation warning which minimises // compile errors, but still fails to compile if ever used. use core::marker::PhantomData; - #[cfg(not(feature = "std"))] use crate::alloc::vec::Vec; + use alloc::vec::Vec; use super::WeightedError; #[derive(Debug)] diff --git a/src/distributions/weighted_index.rs b/src/distributions/weighted_index.rs index ad1c6c722c2..771aeeb940b 100644 --- a/src/distributions/weighted_index.rs +++ b/src/distributions/weighted_index.rs @@ -15,7 +15,7 @@ use core::cmp::PartialOrd; use core::fmt; // Note that this whole module is only imported if feature="alloc" is enabled. -#[cfg(not(feature = "std"))] use crate::alloc::vec::Vec; +use alloc::vec::Vec; #[cfg(feature = "serde1")] use serde::{Serialize, Deserialize}; diff --git a/src/lib.rs b/src/lib.rs index ceb501f4b56..d6b13acdd2d 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -48,7 +48,7 @@ #![deny(missing_docs)] #![deny(missing_debug_implementations)] #![doc(test(attr(allow(unused_variables), deny(warnings))))] -#![cfg_attr(not(feature = "std"), no_std)] +#![no_std] #![cfg_attr(all(feature = "simd_support", feature = "nightly"), feature(stdsimd))] #![cfg_attr(feature = "nightly", feature(slice_partition_at_index))] #![cfg_attr(doc_cfg, feature(doc_cfg))] @@ -58,7 +58,8 @@ clippy::float_cmp )] -#[cfg(all(feature = "alloc", not(feature = "std")))] extern crate alloc; +#[cfg(feature = "std")] extern crate std; +#[cfg(feature = "alloc")] extern crate alloc; #[allow(unused)] macro_rules! trace { ($($x:tt)*) => ( diff --git a/src/rng.rs b/src/rng.rs index 3bff930b4aa..5261b1c14aa 100644 --- a/src/rng.rs +++ b/src/rng.rs @@ -419,7 +419,7 @@ mod test { use super::*; use crate::test::rng; use crate::rngs::mock::StepRng; - #[cfg(all(not(feature = "std"), feature = "alloc"))] use alloc::boxed::Box; + #[cfg(feature = "alloc")] use alloc::boxed::Box; #[test] fn test_fill_bytes_default() { diff --git a/src/rngs/adapter/read.rs b/src/rngs/adapter/read.rs index 7c9018169f5..c855073cd53 100644 --- a/src/rngs/adapter/read.rs +++ b/src/rngs/adapter/read.rs @@ -105,6 +105,8 @@ impl std::error::Error for ReadError { #[cfg(test)] mod test { + use std::{vec, println}; + use super::ReadRng; use crate::RngCore; diff --git a/src/rngs/thread.rs b/src/rngs/thread.rs index ac788afdee0..3a216bc93db 100644 --- a/src/rngs/thread.rs +++ b/src/rngs/thread.rs @@ -10,6 +10,7 @@ use core::cell::UnsafeCell; use core::ptr::NonNull; +use std::thread_local; use super::std::Core; use crate::rngs::adapter::ReseedingRng; diff --git a/src/seq/index.rs b/src/seq/index.rs index 55053e3505e..0a5619c5aae 100644 --- a/src/seq/index.rs +++ b/src/seq/index.rs @@ -10,12 +10,10 @@ #[cfg(feature = "alloc")] use core::slice; -#[cfg(all(feature = "alloc", not(feature = "std")))] -use crate::alloc::vec::{self, Vec}; -#[cfg(feature = "std")] use std::vec; +#[cfg(feature = "alloc")] use alloc::vec::{self, Vec}; // BTreeMap is not as fast in tests, but better than nothing. #[cfg(all(feature = "alloc", not(feature = "std")))] -use crate::alloc::collections::BTreeSet; +use alloc::collections::BTreeSet; #[cfg(feature = "std")] use std::collections::HashSet; #[cfg(feature = "alloc")] @@ -562,8 +560,7 @@ mod test { } } - #[cfg(all(feature = "alloc", not(feature = "std")))] use crate::alloc::vec; - #[cfg(feature = "std")] use std::vec; + #[cfg(feature = "alloc")] use alloc::vec; #[test] fn test_sample_boundaries() { diff --git a/src/seq/mod.rs b/src/seq/mod.rs index 6fac283caf9..fdc0d8acb15 100644 --- a/src/seq/mod.rs +++ b/src/seq/mod.rs @@ -31,7 +31,7 @@ pub mod index; #[cfg(feature = "alloc")] use core::ops::Index; -#[cfg(all(feature = "alloc", not(feature = "std")))] use crate::alloc::vec::Vec; +#[cfg(feature = "alloc")] use alloc::vec::Vec; #[cfg(feature = "alloc")] use crate::distributions::uniform::{SampleBorrow, SampleUniform}; From 6911b8fa5a3a22d39974ed87699a73678e021987 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 22:48:10 +0200 Subject: [PATCH 06/17] Try to fix endianess issue --- rand_core/src/impls.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 6cadcda69df..21592477296 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -117,14 +117,14 @@ pub fn fill_via_u64_chunks(src: &[u64], dest: &mut [u8]) -> (usize, usize) { pub fn next_u32_via_fill(rng: &mut R) -> u32 { let mut buf = [0; 4]; rng.fill_bytes(&mut buf); - u32::from_le_bytes(buf) + u32::from_ne_bytes(buf) } /// Implement `next_u64` via `fill_bytes`, little-endian order. pub fn next_u64_via_fill(rng: &mut R) -> u64 { let mut buf = [0; 8]; rng.fill_bytes(&mut buf); - u64::from_le_bytes(buf) + u64::from_ne_bytes(buf) } // TODO: implement tests for the above From 514e60dcd2c5ca16e57a23c5a326e16575b9543b Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 23:12:04 +0200 Subject: [PATCH 07/17] Simplify tests to address review feedback --- src/distributions/mod.rs | 11 +++++++---- src/rngs/adapter/read.rs | 10 +++++----- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/src/distributions/mod.rs b/src/distributions/mod.rs index 5b1f61d9cd6..db4d19a79ed 100644 --- a/src/distributions/mod.rs +++ b/src/distributions/mod.rs @@ -328,9 +328,8 @@ where pub struct Standard; -#[cfg(all(test, feature = "std"))] +#[cfg(test)] mod tests { - use std::{vec::Vec, println}; use super::{Distribution, Uniform}; use crate::Rng; @@ -339,8 +338,12 @@ mod tests { use crate::distributions::Open01; let mut rng = crate::test::rng(210); let distr = Open01; - let results: Vec = distr.sample_iter(&mut rng).take(100).collect(); - println!("{:?}", results); + let mut iter = Distribution::::sample_iter(distr, &mut rng); + let mut sum: f32 = 0.; + for _ in 0..100 { + sum += iter.next().unwrap(); + } + assert!(sum > 0.); } #[test] diff --git a/src/rngs/adapter/read.rs b/src/rngs/adapter/read.rs index c855073cd53..cdb4f959b19 100644 --- a/src/rngs/adapter/read.rs +++ b/src/rngs/adapter/read.rs @@ -105,7 +105,7 @@ impl std::error::Error for ReadError { #[cfg(test)] mod test { - use std::{vec, println}; + use std::println; use super::ReadRng; use crate::RngCore; @@ -114,9 +114,9 @@ mod test { fn test_reader_rng_u64() { // transmute from the target to avoid endianness concerns. #[rustfmt::skip] - let v = vec![0u8, 0, 0, 0, 0, 0, 0, 1, - 0 , 0, 0, 0, 0, 0, 0, 2, - 0, 0, 0, 0, 0, 0, 0, 3]; + let v = [0u8, 0, 0, 0, 0, 0, 0, 1, + 0, 0, 0, 0, 0, 0, 0, 2, + 0, 0, 0, 0, 0, 0, 0, 3]; let mut rng = ReadRng::new(&v[..]); assert_eq!(rng.next_u64(), 1_u64.to_be()); @@ -126,7 +126,7 @@ mod test { #[test] fn test_reader_rng_u32() { - let v = vec![0u8, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3]; + let v = [0u8, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3]; let mut rng = ReadRng::new(&v[..]); assert_eq!(rng.next_u32(), 1_u32.to_be()); From e97a7e64883a14ee1a00454d696452b5a00ea852 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sat, 1 Aug 2020 23:18:14 +0200 Subject: [PATCH 08/17] WeightedIndex: Fix serde test --- src/distributions/weighted_index.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/distributions/weighted_index.rs b/src/distributions/weighted_index.rs index 771aeeb940b..cd86fbdf9d1 100644 --- a/src/distributions/weighted_index.rs +++ b/src/distributions/weighted_index.rs @@ -246,7 +246,7 @@ mod test { #[cfg(feature = "serde1")] #[test] fn test_weightedindex_serde1() { - let weighted_index = WeightedIndex::new(vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).unwrap(); + let weighted_index = WeightedIndex::new(&[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).unwrap(); let ser_weighted_index = bincode::serialize(&weighted_index).unwrap(); let de_weighted_index: WeightedIndex = From 73422d32fb4e471994d67a24f2fe8d658ff6241f Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sun, 2 Aug 2020 01:13:22 +0200 Subject: [PATCH 09/17] Fix `fill_via_chunks` and add tests --- rand_core/src/impls.rs | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 21592477296..9ff71a45145 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -54,11 +54,12 @@ pub fn fill_bytes_via_next(rng: &mut R, dest: &mut [u8]) { macro_rules! fill_via_chunks { ($src:expr, $dst:expr, $ty:ty, $size:expr) => {{ + debug_assert_eq!(core::mem::size_of::<$ty>(), $size); let chunk_size_u8 = min($src.len() * $size, $dst.len()); let chunk_size = (chunk_size_u8 + $size - 1) / $size; for (&n, chunk) in $src.iter().zip($dst.chunks_mut($size)) { - chunk.clone_from_slice(&n.to_le_bytes()); + chunk.copy_from_slice(&n.to_le_bytes()[..chunk.len()]); } (chunk_size, chunk_size_u8) @@ -127,4 +128,39 @@ pub fn next_u64_via_fill(rng: &mut R) -> u64 { u64::from_ne_bytes(buf) } -// TODO: implement tests for the above +#[cfg(test)] +mod test { + use super::*; + + #[test] + fn test_fill_via_u32_chunks() { + let src = [1, 2, 3]; + let mut dst = [0u8; 11]; + assert_eq!(fill_via_u32_chunks(&src, &mut dst), (3, 11)); + assert_eq!(dst, [1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0]); + + let mut dst = [0u8; 13]; + assert_eq!(fill_via_u32_chunks(&src, &mut dst), (3, 12)); + assert_eq!(dst, [1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 0]); + + let mut dst = [0u8; 5]; + assert_eq!(fill_via_u32_chunks(&src, &mut dst), (2, 5)); + assert_eq!(dst, [1, 0, 0, 0, 2]); + } + + #[test] + fn test_fill_via_u64_chunks() { + let src = [1, 2]; + let mut dst = [0u8; 11]; + assert_eq!(fill_via_u64_chunks(&src, &mut dst), (2, 11)); + assert_eq!(dst, [1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0]); + + let mut dst = [0u8; 17]; + assert_eq!(fill_via_u64_chunks(&src, &mut dst), (2, 16)); + assert_eq!(dst, [1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0]); + + let mut dst = [0u8; 5]; + assert_eq!(fill_via_u64_chunks(&src, &mut dst), (1, 5)); + assert_eq!(dst, [1, 0, 0, 0, 0]); + } +} From 6e27f4132761252f2964a43be34a764754aa677a Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sun, 2 Aug 2020 01:15:42 +0200 Subject: [PATCH 10/17] Simplify macro --- rand_core/src/impls.rs | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 9ff71a45145..0d1ce4ea309 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -53,12 +53,12 @@ pub fn fill_bytes_via_next(rng: &mut R, dest: &mut [u8]) { } macro_rules! fill_via_chunks { - ($src:expr, $dst:expr, $ty:ty, $size:expr) => {{ - debug_assert_eq!(core::mem::size_of::<$ty>(), $size); - let chunk_size_u8 = min($src.len() * $size, $dst.len()); - let chunk_size = (chunk_size_u8 + $size - 1) / $size; + ($src:expr, $dst:expr, $ty:ty) => {{ + const SIZE: usize = core::mem::size_of::<$ty>(); + let chunk_size_u8 = min($src.len() * SIZE, $dst.len()); + let chunk_size = (chunk_size_u8 + SIZE - 1) / SIZE; - for (&n, chunk) in $src.iter().zip($dst.chunks_mut($size)) { + for (&n, chunk) in $src.iter().zip($dst.chunks_mut(SIZE)) { chunk.copy_from_slice(&n.to_le_bytes()[..chunk.len()]); } @@ -97,7 +97,7 @@ macro_rules! fill_via_chunks { /// } /// ``` pub fn fill_via_u32_chunks(src: &[u32], dest: &mut [u8]) -> (usize, usize) { - fill_via_chunks!(src, dest, u32, 4) + fill_via_chunks!(src, dest, u32) } /// Implement `fill_bytes` by reading chunks from the output buffer of a block @@ -111,7 +111,7 @@ pub fn fill_via_u32_chunks(src: &[u32], dest: &mut [u8]) -> (usize, usize) { /// /// See `fill_via_u32_chunks` for an example. pub fn fill_via_u64_chunks(src: &[u64], dest: &mut [u8]) -> (usize, usize) { - fill_via_chunks!(src, dest, u64, 8) + fill_via_chunks!(src, dest, u64) } /// Implement `next_u32` via `fill_bytes`, little-endian order. From e40851a4d6db7c45a03132dbcec9c168ecd73186 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Sun, 2 Aug 2020 02:42:42 +0200 Subject: [PATCH 11/17] Use `chunks_exact_mut` for slightly better performance The results from master, using unsafe code: ``` gen_bytes_chacha12: 2,733,838 ns/iter (+/- 181,694) = 374 MB/s gen_bytes_chacha20: 4,339,602 ns/iter (+/- 237,793) = 235 MB/s gen_bytes_chacha8: 1,918,279 ns/iter (+/- 103,581) = 533 MB/s ``` The results of the new code using `chunks_exact_mut` (this commit): ``` gen_bytes_chacha12: 3,049,147 ns/iter (+/- 220,631) = 335 MB/s gen_bytes_chacha20: 4,645,772 ns/iter (+/- 269,261) = 220 MB/s gen_bytes_chacha8: 2,214,954 ns/iter (+/- 1,745,600) = 462 MB/s ``` The results of using `chunks_mut` (before this commit): ``` gen_bytes_chacha12: 3,492,109 ns/iter (+/- 164,638) = 293 MB/s gen_bytes_chacha20: 5,087,706 ns/iter (+/- 249,219) = 201 MB/s gen_bytes_chacha8: 2,700,197 ns/iter (+/- 524,148) = 379 MB/s ``` --- rand_core/src/impls.rs | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 0d1ce4ea309..2b472216fac 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -58,8 +58,20 @@ macro_rules! fill_via_chunks { let chunk_size_u8 = min($src.len() * SIZE, $dst.len()); let chunk_size = (chunk_size_u8 + SIZE - 1) / SIZE; - for (&n, chunk) in $src.iter().zip($dst.chunks_mut(SIZE)) { - chunk.copy_from_slice(&n.to_le_bytes()[..chunk.len()]); + let mut iter_src = $src.iter(); + let mut iter_chunks = $dst.chunks_exact_mut(SIZE); + let mut n; + loop { + n = iter_src.next(); + if let (Some(n_), Some(chunk)) = (n, iter_chunks.next()) { + chunk.copy_from_slice(&n_.to_le_bytes()); + } else { + break; + } + } + let rem = iter_chunks.into_remainder(); + if let Some(n_) = n { + rem.copy_from_slice(&n_.to_le_bytes()[..rem.len()]); } (chunk_size, chunk_size_u8) From 23e02db8fda3b6a62ba56df76aab7d1323964bde Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Tue, 4 Aug 2020 00:08:23 +0200 Subject: [PATCH 12/17] Check source size and prefer `chunks_exact` --- rand_core/src/le.rs | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/rand_core/src/le.rs b/rand_core/src/le.rs index d7bf57d7c66..fa338928403 100644 --- a/rand_core/src/le.rs +++ b/rand_core/src/le.rs @@ -16,7 +16,8 @@ use core::convert::TryInto; /// Reads unsigned 32 bit integers from `src` into `dst`. #[inline] pub fn read_u32_into(src: &[u8], dst: &mut [u32]) { - for (out, chunk) in dst.iter_mut().zip(src.chunks(4)) { + assert!(4 * src.len() >= dst.len()); + for (out, chunk) in dst.iter_mut().zip(src.chunks_exact(4)) { *out = u32::from_le_bytes(chunk.try_into().unwrap()); } } @@ -24,7 +25,8 @@ pub fn read_u32_into(src: &[u8], dst: &mut [u32]) { /// Reads unsigned 64 bit integers from `src` into `dst`. #[inline] pub fn read_u64_into(src: &[u8], dst: &mut [u64]) { - for (out, chunk) in dst.iter_mut().zip(src.chunks(8)) { + assert!(8 * src.len() >= dst.len()); + for (out, chunk) in dst.iter_mut().zip(src.chunks_exact(8)) { *out = u64::from_le_bytes(chunk.try_into().unwrap()); } } From d74e800f25e8227a0cbe1673a7e205ad0589bf56 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Tue, 4 Aug 2020 07:02:16 +0200 Subject: [PATCH 13/17] Simplify code --- rand_core/src/impls.rs | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 2b472216fac..c847ee0fafc 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -59,19 +59,13 @@ macro_rules! fill_via_chunks { let chunk_size = (chunk_size_u8 + SIZE - 1) / SIZE; let mut iter_src = $src.iter(); - let mut iter_chunks = $dst.chunks_exact_mut(SIZE); - let mut n; - loop { - n = iter_src.next(); - if let (Some(n_), Some(chunk)) = (n, iter_chunks.next()) { - chunk.copy_from_slice(&n_.to_le_bytes()); - } else { - break; - } + let mut chunks = $dst.chunks_exact_mut(SIZE); + for (chunk, n) in (&mut chunks).zip(&mut iter_src) { + chunk.copy_from_slice(&n.to_le_bytes()); } - let rem = iter_chunks.into_remainder(); - if let Some(n_) = n { - rem.copy_from_slice(&n_.to_le_bytes()[..rem.len()]); + let rem = chunks.into_remainder(); + if let Some(n) = iter_src.next() { + rem.copy_from_slice(&n.to_le_bytes()[..rem.len()]); } (chunk_size, chunk_size_u8) From 422bb44730956a2f7669538abee06fbd65bd455f Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Wed, 12 Aug 2020 02:06:28 +0200 Subject: [PATCH 14/17] Restore unsafe `fill_via_chunks` implementation for performance --- rand_core/src/impls.rs | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index c847ee0fafc..53a2d6e4bbf 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -58,6 +58,29 @@ macro_rules! fill_via_chunks { let chunk_size_u8 = min($src.len() * SIZE, $dst.len()); let chunk_size = (chunk_size_u8 + SIZE - 1) / SIZE; + if cfg!(target_endian = "little") { + unsafe { + core::ptr::copy_nonoverlapping( + $src.as_ptr() as *const u8, + $dst.as_mut_ptr(), + chunk_size_u8); + } + } else { + for (&n, chunk) in $src.iter().zip($dst.chunks_mut(SIZE)) { + let tmp = n.to_le(); + let src_ptr = &tmp as *const $ty as *const u8; + unsafe { + core::ptr::copy_nonoverlapping( + src_ptr, + chunk.as_mut_ptr(), + chunk.len()); + } + } + } + + // The following code is a safe replacement, but unfortunately ca. 8% + // slower. + /* let mut iter_src = $src.iter(); let mut chunks = $dst.chunks_exact_mut(SIZE); for (chunk, n) in (&mut chunks).zip(&mut iter_src) { @@ -67,6 +90,7 @@ macro_rules! fill_via_chunks { if let Some(n) = iter_src.next() { rem.copy_from_slice(&n.to_le_bytes()[..rem.len()]); } + */ (chunk_size, chunk_size_u8) }}; From 11439b5a044916cb23a49f26ac9c46dc6f135261 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Thu, 27 Aug 2020 17:11:38 +0200 Subject: [PATCH 15/17] Delete dead code --- rand_core/src/impls.rs | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/rand_core/src/impls.rs b/rand_core/src/impls.rs index 53a2d6e4bbf..79258364618 100644 --- a/rand_core/src/impls.rs +++ b/rand_core/src/impls.rs @@ -58,6 +58,8 @@ macro_rules! fill_via_chunks { let chunk_size_u8 = min($src.len() * SIZE, $dst.len()); let chunk_size = (chunk_size_u8 + SIZE - 1) / SIZE; + // The following can be replaced with safe code, but unfortunately it's + // ca. 8% slower. if cfg!(target_endian = "little") { unsafe { core::ptr::copy_nonoverlapping( @@ -78,20 +80,6 @@ macro_rules! fill_via_chunks { } } - // The following code is a safe replacement, but unfortunately ca. 8% - // slower. - /* - let mut iter_src = $src.iter(); - let mut chunks = $dst.chunks_exact_mut(SIZE); - for (chunk, n) in (&mut chunks).zip(&mut iter_src) { - chunk.copy_from_slice(&n.to_le_bytes()); - } - let rem = chunks.into_remainder(); - if let Some(n) = iter_src.next() { - rem.copy_from_slice(&n.to_le_bytes()[..rem.len()]); - } - */ - (chunk_size, chunk_size_u8) }}; } From 0913bc70965ea81fba4bdb20976ad689c31466ec Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Thu, 27 Aug 2020 17:13:37 +0200 Subject: [PATCH 16/17] Slightly improve test --- src/distributions/mod.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/distributions/mod.rs b/src/distributions/mod.rs index db4d19a79ed..0d69d28edb4 100644 --- a/src/distributions/mod.rs +++ b/src/distributions/mod.rs @@ -343,7 +343,7 @@ mod tests { for _ in 0..100 { sum += iter.next().unwrap(); } - assert!(sum > 0.); + assert!(0. < sum && sum < 100.); } #[test] From 2bc54d05df314905bf337538a499e4aece5647f0 Mon Sep 17 00:00:00 2001 From: Vinzent Steinberg Date: Thu, 27 Aug 2020 18:24:33 +0200 Subject: [PATCH 17/17] Update changelog --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f1a6ae8bf27..9b8b3236eae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ You may also find the [Upgrade Guide](https://rust-random.github.io/book/update. - impl PartialEq+Eq for StdRng, SmallRng, and StepRng (#975) - Added a `serde1` feature and added Serialize/Deserialize to `UniformInt` and `WeightedIndex` (#974) - Document types supported by `random` (#994) +- Implement weighted sampling without replacement (#976, #1013) ### Changes - `gen_range(a, b)` was replaced with `gen_range(a..b)`, and `gen_range(a..=b)` @@ -20,6 +21,7 @@ You may also find the [Upgrade Guide](https://rust-random.github.io/book/update. - Replace `AsByteSliceMut` with `Fill` (#940) - Move alias method for `WeightedIndex` to `rand_distr` (#945) - `Alphanumeric` samples bytes instead of chars (#935) +- The minimum supported Rust version is now 1.36 (#1011) - Better NaN handling for `WeightedIndex` (#1005) - Implement `IntoIterator` for `IndexVec`, replacing the `into_iter` method (#1007) - Reduce packaged crate size (#983)