Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement io-uring for actix-files #2408

Merged
merged 36 commits into from
Nov 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
bf72454
implement io-uring for actix-files
fakeshadow Oct 13, 2021
0932884
fix clippy
fakeshadow Oct 13, 2021
3f72074
duplicate one test for io-uring feature
fakeshadow Oct 13, 2021
3e578e8
fix foreign File::open API
fakeshadow Oct 13, 2021
9ffabe9
fix raw fd drop.
fakeshadow Oct 14, 2021
a751be7
reduce heap allocation when chunked streaming.
fakeshadow Oct 16, 2021
5d3d1fb
fix clippy warning
fakeshadow Oct 16, 2021
d521c57
Merge branch 'master' into feat/io-uring
fakeshadow Oct 16, 2021
200cf0c
add re-used buffer for io-uring feature
fakeshadow Oct 17, 2021
8bc92b7
remove NamedFile::open from io-uring feature
fakeshadow Oct 17, 2021
cba70f6
fix a couple of doc test by using async api
fakeshadow Oct 17, 2021
e2ac85a
fix tests
fakeshadow Oct 17, 2021
159428a
remove actix-util as dep
fakeshadow Oct 18, 2021
5bf874e
fix test
fakeshadow Oct 18, 2021
c284b4c
reduce clones called on pathbuf
fakeshadow Oct 18, 2021
31bbd73
Merge branch 'master' into feat/io-uring
robjtede Oct 20, 2021
b9343bc
Merge branch 'master' into feat/io-uring
fakeshadow Oct 20, 2021
8c8746b
Merge branch 'master' into feat/io-uring
fakeshadow Oct 20, 2021
f8032ad
fix double drop on error of getting file metadata
fakeshadow Oct 21, 2021
1cc191d
Merge branch 'master' into feat/io-uring
fakeshadow Oct 21, 2021
c1ab92f
temporary override actix-rt
fakeshadow Nov 6, 2021
3de6d67
Merge branch 'master' into feat/io-uring
fakeshadow Nov 6, 2021
32d2154
Merge branch 'master' into feat/io-uring
fakeshadow Nov 9, 2021
aa5ef14
remove actix-rt/io-uring feature requirement
fakeshadow Nov 9, 2021
24f764f
skip io-uring feature in cargo hack
fakeshadow Nov 9, 2021
d30c861
Merge branch 'master' into feat/io-uring
robjtede Nov 15, 2021
36af057
Merge branch 'master' into feat/io-uring
fakeshadow Nov 16, 2021
68a8fce
Merge branch 'master' into feat/io-uring
robjtede Nov 19, 2021
0cd921b
set up CI manually for io-uring testing
robjtede Nov 21, 2021
7557f07
make sure ulimit is increase for uring tests
robjtede Nov 21, 2021
3dfa8b0
Merge remote-tracking branch 'origin/master' into feat/io-uring
robjtede Nov 21, 2021
e9554bf
rename feature and add changelog
robjtede Nov 21, 2021
452331b
fix ci
robjtede Nov 21, 2021
1f28bea
run test servers in io-uring rt friendly way
robjtede Nov 21, 2021
fae8956
fix ci, don't run doctests
robjtede Nov 21, 2021
a717ed7
update changelogs
robjtede Nov 22, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
22 changes: 10 additions & 12 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
[alias]
chk = "check --workspace --all-features --tests --examples --bins"
lint = "clippy --workspace --all-features --tests --examples --bins"
ci-min = "hack check --workspace --no-default-features"
ci-min-test = "hack check --workspace --no-default-features --tests --examples"
ci-default = "check --workspace --bins --tests --examples"
ci-full = "check --workspace --all-features --bins --tests --examples"
ci-test = "test --workspace --all-features --lib --tests --no-fail-fast -- --nocapture"
ci-doctest = "test --workspace --all-features --doc --no-fail-fast -- --nocapture"
lint = "clippy --workspace --tests --examples --bins -- -Dclippy::todo"
lint-all = "clippy --workspace --all-features --tests --examples --bins -- -Dclippy::todo"

# lib checking
ci-check-min = "hack --workspace check --no-default-features"
ci-check-default = "hack --workspace check"
ci-check-all-feature-powerset="hack --workspace --feature-powerset --skip=__compress,io-uring check"
ci-check-all-feature-powerset-linux="hack --workspace --feature-powerset --skip=__compress check"

ci-feature-powerset-check-no-tls="hack --workspace --feature-powerset --skip=__compress,rustls,openssl check"
ci-feature-powerset-check-rustls="hack --workspace --feature-powerset --features=rustls --skip=__compress,openssl check"
ci-feature-powerset-check-openssl="hack --workspace --feature-powerset --features=openssl --skip=__compress,rustls check"
ci-feature-powerset-check-all="hack --workspace --feature-powerset --skip=__compress check"
# testing
ci-doctest = "test --workspace --all-features --doc --no-fail-fast -- --nocapture"
53 changes: 32 additions & 21 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,26 +62,34 @@ jobs:

- name: check minimal
uses: actions-rs/cargo@v1
with: { command: ci-min }

- name: check minimal + tests
uses: actions-rs/cargo@v1
with: { command: ci-min-test }
with: { command: ci-check-min }

- name: check default
uses: actions-rs/cargo@v1
with: { command: ci-default }

- name: check full
uses: actions-rs/cargo@v1
with: { command: ci-full }
with: { command: ci-check-default }

- name: tests
uses: actions-rs/cargo@v1
timeout-minutes: 60
with:
command: ci-test
args: --skip=test_reading_deflate_encoding_large_random_rustls
run: |
cargo test --lib --tests -p=actix-router --all-features
cargo test --lib --tests -p=actix-http --all-features
cargo test --lib --tests -p=actix-web --features=rustls,openssl -- --skip=test_reading_deflate_encoding_large_random_rustls
cargo test --lib --tests -p=actix-web-codegen --all-features
cargo test --lib --tests -p=awc --all-features
cargo test --lib --tests -p=actix-http-test --all-features
cargo test --lib --tests -p=actix-test --all-features
cargo test --lib --tests -p=actix-files
cargo test --lib --tests -p=actix-multipart --all-features
cargo test --lib --tests -p=actix-web-actors --all-features

- name: tests (io-uring)
if: matrix.target.os == 'ubuntu-latest'
timeout-minutes: 60
run: >
sudo bash -c "ulimit -Sl 512
&& ulimit -Hl 512
&& PATH=$PATH:/usr/share/rust/.cargo/bin
&& RUSTUP_TOOLCHAIN=${{ matrix.version }} cargo test --lib --tests -p=actix-files --all-features"

- name: Clear the cargo caches
run: |
Expand Down Expand Up @@ -114,9 +122,12 @@ jobs:
args: cargo-hack

- name: check feature combinations
# if: github.ref == 'refs/heads/master'
uses: actions-rs/cargo@v1
with: { command: ci-feature-powerset-check-all }
with: { command: ci-check-all-feature-powerset }

- name: check feature combinations
uses: actions-rs/cargo@v1
with: { command: ci-check-all-feature-powerset-linux }

coverage:
name: coverage
Expand Down Expand Up @@ -166,11 +177,11 @@ jobs:
- name: Cache Dependencies
uses: Swatinem/rust-cache@v1.3.0

- name: Install cargo-hack
uses: actions-rs/cargo@v1
with:
command: install
args: cargo-hack
# - name: Install cargo-hack
# uses: actions-rs/cargo@v1
# with:
# command: install
# args: cargo-hack

- name: doc tests
uses: actions-rs/cargo@v1
Expand Down
5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,13 @@ rustls = ["actix-http/rustls", "actix-tls/accept", "actix-tls/rustls"]
# Don't rely on these whatsoever. They may disappear at anytime.
__compress = []

# io-uring feature only avaiable for Linux OSes.
experimental-io-uring = ["actix-server/io-uring"]

[dependencies]
actix-codec = "0.4.1"
actix-macros = "0.2.3"
actix-rt = "2.2"
actix-rt = "2.3"
actix-server = "2.0.0-beta.9"
actix-service = "2.0.0"
actix-utils = "3.0.0"
Expand Down
6 changes: 6 additions & 0 deletions actix-files/CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
# Changes

## Unreleased - 2021-xx-xx
* Add crate feature `experimental-io-uring`, enabling async file I/O to be utilized. This feature is only available on Linux OSes with recent kernel versions. This feature is semver-exempt. [#2408]
* Add `NamedFile::open_async`. [#2408]
* Fix 304 Not Modified responses to omit the Content-Length header, as per the spec. [#2453]
* The `Responder` impl for `NamedFile` now has a boxed future associated type. [#2408]
* The `Service` impl for `NamedFileService` now has a boxed future associated type. [#2408]
* Add `impl Clone` for `FilesService`. [#2408]

[#2408]: https://github.com/actix/actix-web/pull/2408
[#2453]: https://github.com/actix/actix-web/pull/2453


Expand Down
7 changes: 6 additions & 1 deletion actix-files/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,13 @@ edition = "2018"
name = "actix_files"
path = "src/lib.rs"

[features]
experimental-io-uring = ["actix-web/experimental-io-uring", "tokio-uring"]

[dependencies]
actix-web = { version = "4.0.0-beta.11", default-features = false }
actix-http = "3.0.0-beta.12"
actix-service = "2.0.0"
actix-utils = "3.0.0"

askama_escape = "0.10"
bitflags = "1"
Expand All @@ -30,6 +32,9 @@ log = "0.4"
mime = "0.3"
mime_guess = "2.0.1"
percent-encoding = "2.1"
pin-project-lite = "0.2.7"

tokio-uring = { version = "0.1", optional = true }

[dev-dependencies]
actix-rt = "2.2"
Expand Down