Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Linux on MIPS and ARMv5TE #39

Merged
merged 9 commits into from Sep 10, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/CI.yml
Expand Up @@ -68,6 +68,15 @@ jobs:
env:
RUSTFLAGS: '--cfg skeptic'

- name: Run tests (future, without atomic64)
uses: actions-rs/cargo@v1
if: ${{ matrix.rust != '1.45.2' }}
with:
command: test
args: --release --no-default-features --features future
env:
RUSTFLAGS: '--cfg skeptic'

- name: Run UI tests (future, trybuild)
uses: actions-rs/cargo@v1
if: ${{ matrix.rust == 'stable' }}
Expand Down
2 changes: 2 additions & 0 deletions .vscode/settings.json
@@ -1,11 +1,13 @@
{
"rust-analyzer.cargo.features": ["future"],
"cSpell.words": [
"aarch",
"actix",
"ahash",
"benmanes",
"CLFU",
"clippy",
"compat",
"cpus",
"deqs",
"Deque",
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.md
@@ -1,5 +1,15 @@
# Moka — Change Log

## Version 0.5.3

### Added

- Add support for some 32-bit platforms where `std::sync::atomic::AtomicU64` is not
provided. (e.g. `armv5te-unknown-linux-musleabi` or `mips-unknown-linux-musl`)
- On these platforms, you will need to disable the default features of Moka.
See [the relevant section][resolving-error-on-32bit] of the README.


## Version 0.5.2

### Fixed
Expand Down Expand Up @@ -104,6 +114,9 @@

[caffeine-git]: https://github.com/ben-manes/caffeine

[resolving-error-on-32bit]: https://github.com/moka-rs/moka#resolving-compile-errors-on-some-32-bit-platforms

[gh-issue-0038]: https://github.com/moka-rs/moka/issues/38/
[gh-pull-0033]: https://github.com/moka-rs/moka/pull/33/
[gh-issue-0031]: https://github.com/moka-rs/moka/issues/31/
[gh-pull-0030]: https://github.com/moka-rs/moka/pull/30/
Expand Down
14 changes: 11 additions & 3 deletions Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "moka"
version = "0.5.2"
version = "0.5.3"
authors = ["Tatsuya Kawano <tatsuya@hibaridb.org>"]
edition = "2018"

Expand All @@ -20,23 +20,31 @@ build = "build.rs"
features = ["future"]

[features]
default = []
default = ["atomic64"]

# Enable this feature to use `moka::future::Cache`.
future = ["async-io", "async-lock"]

# This feature is enabled by default. Disable it when the target platform does not
# support `std::sync::atomic::AtomicU64`. (e.g. `armv5te-unknown-linux-musleabi`
# or `mips-unknown-linux-musl`)
# https://github.com/moka-rs/moka#resolving-compile-errors-on-some-32-bit-platforms
atomic64 = ["quanta"]

[dependencies]
crossbeam-channel = "0.5"
moka-cht = "0.4.2"
num_cpus = "1.13"
once_cell = "1.7"
parking_lot = "0.11"
quanta = "0.9"
scheduled-thread-pool = "0.2"
thiserror = "1.0"
uuid = { version = "0.8", features = ["v4"] }

# Optional dependencies
async-io = { version = "1.4", optional = true }
async-lock = { version = "2.4", optional = true }
quanta = { version = "0.9", optional = true }

[dev-dependencies]
actix-rt2 = { package = "actix-rt", version = "2", default-features = false }
Expand Down
61 changes: 53 additions & 8 deletions README.md
Expand Up @@ -317,14 +317,15 @@ available on crates.io, such as the [aHash][ahash-crate] crate.

This crate's minimum supported Rust versions (MSRV) are the followings:

| Enabled Feature | MSRV |
|:---------------------|:------------|
| no feature (default) | Rust 1.45.2 |
| `future` | Rust 1.46.0 |

If no feature is enabled, MSRV will be updated conservatively. When using other
features, like `future`, MSRV might be updated more frequently, up to the latest
stable. In both cases, increasing MSRV is _not_ considered a semver-breaking
| Feature | Enabled by default? | MSRV |
|:-----------|:-------------------:|:-----------:|
| no feature | | Rust 1.45.2 |
| `atomic64` | yes | Rust 1.45.2 |
| `future` | | Rust 1.46.0 |

If only the default features are enabled, MSRV will be updated conservatively. When
using other features, like `future`, MSRV might be updated more frequently, up to the
latest stable. In both cases, increasing MSRV is _not_ considered a semver-breaking
change.

<!--
Expand All @@ -334,6 +335,50 @@ change.
-->


## Resolving Compile Errors on Some 32-bit Platforms

On some 32-bit target platforms including the followings, you may encounter compile
errors in quanta crate and Moka itself.

- `armv5te-unknown-linux-musleabi`
- `mips-unknown-linux-musl`
- `mipsel-unknown-linux-musl`

```console
error[E0599]: no method named `fetch_add` found for struct `Arc<AtomicCell<u64>>` in the current scope
--> ... /quanta-0.9.2/src/mock.rs:48:21
|
48 | self.offset.fetch_add(amount.into_nanos());
| ^^^^^^^^^ method not found in `Arc<AtomicCell<u64>>`
```

```console
error[E0432]: unresolved import `std::sync::atomic::AtomicU64`
--> ... /moka-0.5.3/src/sync.rs:10:30
|
10 | atomic::{AtomicBool, AtomicU64, Ordering},
| ^^^^^^^^^
| |
| no `AtomicU64` in `sync::atomic`
```

These errors occur because `std::sync::atomic::AtomicU64` type is not provided on
these platforms but both quanta and Moka use it.

You can resolve the errors by disabling the default features of Moka. Edit your
Cargo.toml to add `default-features = false` option to the dependency declaration.

```toml:Cargo.toml
[dependencies]
moka = { version = "0.5", default-feautures = false }
# Or
moka = { version = "0.5", default-feautures = false, features = ["future"] }
```

This will remove quanta from the dependencies and enable a fall-back implementation
in Moka, so it will compile.


## Developing Moka

**Running All Tests**
Expand Down
20 changes: 10 additions & 10 deletions src/common.rs
@@ -1,22 +1,22 @@
use quanta::Instant;

pub(crate) mod deque;
pub(crate) mod error;
pub(crate) mod frequency_sketch;
pub(crate) mod thread_pool;
pub(crate) mod unsafe_weak_pointer;

// targe_has_atomic is more convenient but yet unstable (Rust 1.55)
// https://github.com/rust-lang/rust/issues/32976
// #[cfg_attr(target_has_atomic = "64", path = "common/time_quanta.rs")]

#[cfg_attr(feature = "atomic64", path = "common/time_quanta.rs")]
#[cfg_attr(not(feature = "atomic64"), path = "common/time_compat.rs")]
pub(crate) mod time;

use time::Instant;

pub(crate) trait AccessTime {
fn last_accessed(&self) -> Option<Instant>;
fn set_last_accessed(&mut self, timestamp: Instant);
fn last_modified(&self) -> Option<Instant>;
fn set_last_modified(&mut self, timestamp: Instant);
}

pub(crate) fn u64_to_instant(ts: u64) -> Option<Instant> {
if ts == u64::MAX {
None
} else {
Some(unsafe { std::mem::transmute(ts) })
}
}
5 changes: 4 additions & 1 deletion src/common/thread_pool.rs
Expand Up @@ -53,7 +53,10 @@ impl ThreadPoolRegistry {
// and insert a new pool.
let mut pools = REGISTRY.pools.write();
pools.entry(name).or_insert_with(|| {
let num_threads = num_cpus::get();
// Some platforms may return 0. In that case, use 1.
// https://github.com/moka-rs/moka/pull/39#issuecomment-916888859
// https://github.com/rust-lang/futures-rs/pull/1835
let num_threads = num_cpus::get().max(1);
let pool =
ScheduledThreadPool::with_name(name.thread_name_template(), num_threads);
let t_pool = ThreadPool {
Expand Down
105 changes: 105 additions & 0 deletions src/common/time_compat.rs
@@ -0,0 +1,105 @@
use std::{
cmp::Ordering,
ops::Add,
sync::Arc,
time::{Duration, Instant as StdInstant},
};

use parking_lot::RwLock;

#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) struct Instant(StdInstant);

impl Instant {
pub(crate) fn now() -> Self {
Self(StdInstant::now())
}
}

impl Add<Duration> for Instant {
type Output = Instant;

fn add(self, other: Duration) -> Self::Output {
let instant = self
.0
.checked_add(other)
.expect("overflow when adding duration to instant");
Self(instant)
}
}

impl PartialOrd for Instant {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}

impl Ord for Instant {
fn cmp(&self, other: &Self) -> Ordering {
self.0.cmp(&other.0)
}
}

pub(crate) struct AtomicInstant {
instant: RwLock<Option<Instant>>,
}

impl Default for AtomicInstant {
fn default() -> Self {
Self {
instant: RwLock::new(None),
}
}
}

impl AtomicInstant {
pub(crate) fn reset(&self) {
*self.instant.write() = None;
}

pub(crate) fn is_set(&self) -> bool {
self.instant.read().is_some()
}

pub(crate) fn instant(&self) -> Option<Instant> {
*self.instant.read()
}

pub(crate) fn set_instant(&self, instant: Instant) {
*self.instant.write() = Some(instant);
}
}

pub(crate) struct Clock(Arc<Mock>);

impl Clock {
#[cfg(test)]
pub(crate) fn mock() -> (Clock, Arc<Mock>) {
let mock = Arc::new(Mock::default());
let clock = Clock(Arc::clone(&mock));
(clock, mock)
}

pub(crate) fn now(&self) -> Instant {
Instant(*self.0.now.read())
}
}

pub(crate) struct Mock {
now: RwLock<StdInstant>,
}

impl Default for Mock {
fn default() -> Self {
Self {
now: RwLock::new(StdInstant::now()),
}
}
}

#[cfg(test)]
impl Mock {
pub(crate) fn increment(&self, amount: Duration) {
*self.now.write() += amount;
}
}
42 changes: 42 additions & 0 deletions src/common/time_quanta.rs
@@ -0,0 +1,42 @@
use std::sync::atomic::{AtomicU64, Ordering};

pub(crate) type Instant = quanta::Instant;
pub(crate) type Clock = quanta::Clock;

#[cfg(test)]
pub(crate) type Mock = quanta::Mock;

pub(crate) struct AtomicInstant {
instant: AtomicU64,
}

impl Default for AtomicInstant {
fn default() -> Self {
Self {
instant: AtomicU64::new(std::u64::MAX),
}
}
}

impl AtomicInstant {
pub(crate) fn reset(&self) {
self.instant.store(std::u64::MAX, Ordering::Release);
}

pub(crate) fn is_set(&self) -> bool {
self.instant.load(Ordering::Acquire) != u64::MAX
}

pub(crate) fn instant(&self) -> Option<Instant> {
let ts = self.instant.load(Ordering::Acquire);
if ts == u64::MAX {
None
} else {
Some(unsafe { std::mem::transmute(ts) })
}
}

pub(crate) fn set_instant(&self, instant: Instant) {
self.instant.store(instant.as_u64(), Ordering::Release);
}
}
5 changes: 2 additions & 3 deletions src/future/cache.rs
Expand Up @@ -749,7 +749,7 @@ where
self.base.reconfigure_for_testing();
}

fn set_expiration_clock(&self, clock: Option<quanta::Clock>) {
fn set_expiration_clock(&self, clock: Option<crate::common::time::Clock>) {
self.base.set_expiration_clock(clock);
}
}
Expand All @@ -758,10 +758,9 @@ where
#[cfg(test)]
mod tests {
use super::{Cache, ConcurrentCacheExt};
use crate::future::CacheBuilder;
use crate::{common::time::Clock, future::CacheBuilder};

use async_io::Timer;
use quanta::Clock;
use std::time::Duration;

#[tokio::test]
Expand Down