Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3 case studies for -Zmiri-tag-raw-pointers #1936

Closed
dtolnay opened this issue Dec 12, 2021 · 9 comments
Closed

3 case studies for -Zmiri-tag-raw-pointers #1936

dtolnay opened this issue Dec 12, 2021 · 9 comments
Labels
A-aliasing Area: This affects the aliasing model (Stacked/Tree Borrows) C-question Category: Not necessarily a bug, but someone asking for support

Comments

@dtolnay
Copy link
Member

dtolnay commented Dec 12, 2021

Following @saethlin's and @5225225's PRs dtolnay/ryu#37 and dtolnay/ryu#38, I've been trying -Zmiri-tag-raw-pointers on some of my repos containing unsafe code. So far the findings mostly line up with my expectations — I just had to fix a few easy cases similar to the PR above (array.get_unchecked(i)array.as_ptr().offset(i)).

I found 3 cases of more complicated data structures that I would love to get working with -Zmiri-tag-raw-pointers. I am opening this issue not primarily because I want someone to come fix my broken code, but in case these are helpful real-world data structures to inform the implementation of miri-tag-raw-pointers or the resolution of rust-lang/unsafe-code-guidelines#148 and rust-lang/unsafe-code-guidelines#194, which looked the most relevant to me. Please go ahead and close if you are chock full of controvertible code already and these are not going to be useful.



https://github.com/dtolnay/synTokenBuffer

MIRIFLAGS=-Zmiri-tag-raw-pointers cargo miri test --all-features --test test_item
error: Undefined Behavior: no item granting read access to tag <408023> at alloc154479+0x40 found in borrow stack.
   --> /git/syn/src/buffer.rs:191:19
    |
191 |         while let Entry::End(exit) = *ptr {
    |                   ^^^^^^^^^^^^^^^^ no item granting read access to tag <408023> at alloc154479+0x40 found in borrow stack.
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
            
    = note: inside `syn::buffer::Cursor::create` at /git/syn/src/buffer.rs:191:19
    = note: inside `syn::buffer::Cursor::bump` at /git/syn/src/buffer.rs:214:9
    = note: inside `syn::buffer::Cursor::group` at /git/syn/src/buffer.rs:256:66
    = note: inside closure at /git/syn/src/group.rs:75:46
    = note: inside `syn::parse::ParseBuffer::step::<[closure@syn::group::parse_delimited::{closure#0}], (proc_macro2::Span, syn::parse::ParseBuffer)>` at /git/syn/src/parse.rs:1024:28
    = note: inside `syn::group::parse_delimited` at /git/syn/src/group.rs:74:5
    = note: inside `syn::group::parse_brackets` at /git/syn/src/group.rs:56:5
    = note: inside `syn::attr::parsing::single_parse_outer` at /git/syn/src/group.rs:272:15
    = note: inside `syn::parse::ParseBuffer::call::<syn::Attribute>` at /git/syn/src/parse.rs:505:9
    = note: inside `syn::Attribute::parse_outer` at /git/syn/src/attr.rs:241:24
    = note: inside `syn::parse::ParseBuffer::call::<std::vec::Vec<syn::Attribute>>` at /git/syn/src/parse.rs:505:9
    = note: inside `syn::item::parsing::<impl syn::parse::Parse for syn::Item>::parse` at /git/syn/src/item.rs:994:29
    = note: inside `<for<'r> fn(&'r syn::parse::ParseBuffer<'r>) -> std::result::Result<syn::Item, syn::Error> {<syn::Item as syn::parse::Parse>::parse} as std::ops::FnOnce<(&syn::parse::ParseBuffer,)>>::call_once - shim(for<'r> fn(&'r syn::parse::ParseBuffer<'r>) -> std::result::Result<syn::Item, syn::Error> {<syn::Item as syn::parse::Parse>::parse})` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<for<'r> fn(&'r syn::parse::ParseBuffer<'r>) -> std::result::Result<syn::Item, syn::Error> {<syn::Item as syn::parse::Parse>::parse} as syn::parse::Parser>::parse2` at /git/syn/src/parse.rs:1209:20
    = note: inside `syn::parse2::<syn::Item>` at /git/syn/src/lib.rs:913:5
note: inside `<proc_macro2::TokenStream as macros::Tokens>::parse::<syn::Item>` at tests/macros/mod.rs:79:9
   --> tests/macros/mod.rs:79:9
    |
79  |         syn::parse2(self)
    |         ^^^^^^^^^^^^^^^^^
note: inside `test_macro_variable_attr` at tests/macros/mod.rs:40:21
   --> tests/macros/mod.rs:40:21
    |
40  |           let $expr = crate::macros::Tokens::parse::<$t>($expr).unwrap();
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
   ::: tests/test_item.rs:20:5
    |
20  | /     snapshot!(tokens as Item, @r###"
21  | |     Item::Fn {
22  | |         attrs: [
23  | |             Attribute {
...   |
43  | |     }
44  | |     "###);
    | |_________- in this macro invocation
note: inside closure at tests/test_item.rs:10:1
   --> tests/test_item.rs:10:1
    |
9   |   #[test]
    |   ------- in this procedural macro expansion
10  | / fn test_macro_variable_attr() {
11  | |     // mimics the token stream corresponding to `$attr fn f() {}`
12  | |     let tokens = TokenStream::from_iter(vec![
13  | |         TokenTree::Group(Group::new(Delimiter::None, quote! { #[test] })),
...   |
44  | |     "###);
45  | | }
    | |_^
    = note: inside `<[closure@tests/test_item.rs:10:1: 45:2] as std::ops::FnOnce<()>>::call_once - shim` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:585:5
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:576:30
    = note: inside `<[closure@test::run_test::{closure#2}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1811:9
    = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
    = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
    = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
    = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
    = note: inside `test::run_test_in_process` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:608:18
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:500:39
    = note: inside `test::run_test::run_test_inner` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:538:13
    = note: inside `test::run_test` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:572:28
    = note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:313:17
    = note: inside `test::run_tests_console` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/console.rs:290:5
    = note: inside `test::test_main` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:124:15
    = note: inside `test::test_main_static` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:143:5
    = note: inside `main`
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:123:18
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
    = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
    = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
    = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
    = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
    = note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
    = note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
    = note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
    = note: inside `std::rt::lang_start_internal` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
    = note: inside `std::rt::lang_start::<()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
    = note: this error originates in the macro `snapshot_impl` (in Nightly builds, run with -Z macro-backtrace for more info)

https://github.com/dtolnay/semverIdentifier

MIRIFLAGS=-Zmiri-tag-raw-pointers cargo miri test --test test_identifier
error: Undefined Behavior: no item granting read access to tag <untagged> at alloc64891 found in borrow stack.
   --> /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:701:9
    |
701 |         copy_nonoverlapping(src, tmp.as_mut_ptr(), 1);
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no item granting read access to tag <untagged> at alloc64891 found in borrow stack.
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
            
    = note: inside `std::ptr::read::<[u8; 2]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:701:9
    = note: inside `semver::identifier::decode_len` at /git/semver/src/identifier.rs:301:36
    = note: inside `<semver::identifier::Identifier as std::ops::Drop>::drop` at /git/semver/src/identifier.rs:200:28
    = note: inside `std::ptr::drop_in_place::<semver::identifier::Identifier> - shim(Some(semver::identifier::Identifier))` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<semver::Prerelease> - shim(Some(semver::Prerelease))` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
note: inside `test_eq` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/macros/mod.rs:97:5
   --> tests/test_identifier.rs:41:5
    |
41  |     assert_ne!(prerelease("aaaaaaaaa"), prerelease("a"));
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure at tests/test_identifier.rs:37:1
   --> tests/test_identifier.rs:37:1
    |
36  |   #[test]
    |   ------- in this procedural macro expansion
37  | / fn test_eq() {
38  | |     assert_eq!(prerelease("-"), prerelease("-"));
39  | |     assert_ne!(prerelease("a"), prerelease("aa"));
40  | |     assert_ne!(prerelease("aa"), prerelease("a"));
...   |
44  | |     assert_ne!(build_metadata("1"), build_metadata("001"));
45  | | }
    | |_^
    = note: inside `<[closure@tests/test_identifier.rs:37:1: 45:2] as std::ops::FnOnce<()>>::call_once - shim` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:585:5
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:576:30
    = note: inside `<[closure@test::run_test::{closure#2}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1811:9
    = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
    = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
    = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
    = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
    = note: inside `test::run_test_in_process` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:608:18
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:500:39
    = note: inside `test::run_test::run_test_inner` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:538:13
    = note: inside `test::run_test` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:572:28
    = note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:313:17
    = note: inside `test::run_tests_console` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/console.rs:290:5
    = note: inside `test::test_main` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:124:15
    = note: inside `test::test_main_static` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:143:5
    = note: inside `main`
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:123:18
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
    = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
    = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
    = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
    = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
    = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
    = note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
    = note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
    = note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
    = note: inside `std::rt::lang_start_internal` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
    = note: inside `std::rt::lang_start::<()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
    = note: this error originates in the macro `assert_ne` (in Nightly builds, run with -Z macro-backtrace for more info)

https://github.com/dtolnay/miniserdeDeserializer

cargo miri test --test test_derive
error: Undefined Behavior: trying to reborrow for SharedReadWrite at alloc64094, but parent tag <165391> does not have an appropriate item in the borrow stack
  --> /git/miniserde/src/json/de.rs:88:17
   |
88 |                 visitor.string(s)?;
   |                 ^^^^^^^^^^^^^^^^^ trying to reborrow for SharedReadWrite at alloc64094, but parent tag <165391> does not have an appropriate item in the borrow stack
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
           
   = note: inside `miniserde::json::de::from_str_impl` at /git/miniserde/src/json/de.rs:88:17
   = note: inside `miniserde::json::from_str::<Example>` at /git/miniserde/src/json/de.rs:32:5
note: inside `test_de` at tests/test_derive.rs:27:27
  --> tests/test_derive.rs:27:27
   |
27 |     let actual: Example = json::from_str(j).unwrap();
   |                           ^^^^^^^^^^^^^^^^^
note: inside closure at tests/test_derive.rs:25:1
  --> tests/test_derive.rs:25:1
   |
24 |   #[test]
   |   ------- in this procedural macro expansion
25 | / fn test_de() {
26 | |     let j = r#" {"x": "X", "t1": "A", "t2": "renamedB", "n": {"y": ["Y", "Y"]}} "#;
27 | |     let actual: Example = json::from_str(j).unwrap();
28 | |     let expected = Example {
...  |
37 | |     assert_eq!(actual, expected);
38 | | }
   | |_^
   = note: inside `<[closure@tests/test_derive.rs:25:1: 38:2] as std::ops::FnOnce<()>>::call_once - shim` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:585:5
   = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:576:30
   = note: inside `<[closure@test::run_test::{closure#2}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1811:9
   = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
   = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
   = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
   = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
   = note: inside `test::run_test_in_process` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:608:18
   = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:500:39
   = note: inside `test::run_test::run_test_inner` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:538:13
   = note: inside `test::run_test` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:572:28
   = note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:313:17
   = note: inside `test::run_tests_console` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/console.rs:290:5
   = note: inside `test::test_main` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:124:15
   = note: inside `test::test_main_static` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:143:5
   = note: inside `main`
   = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:123:18
   = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
   = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
   = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
   = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
   = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
   = note: inside closure at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
   = note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:406:40
   = note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:370:19
   = note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:133:14
   = note: inside `std::rt::lang_start_internal` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
   = note: inside `std::rt::lang_start::<()>` at /home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
   = note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

@RalfJung RalfJung added A-aliasing Area: This affects the aliasing model (Stacked/Tree Borrows) C-question Category: Not necessarily a bug, but someone asking for support labels Dec 16, 2021
@RalfJung
Copy link
Member

RalfJung commented Dec 16, 2021

Thanks for sharing!

So do all of these involve self-referential structs or 'owning-ref'-style pointers? (I am asking since you mentioned rust-lang/unsafe-code-guidelines#148 and rust-lang/unsafe-code-guidelines#194.)

@saethlin
Copy link
Member

I don't have any comment on the syn example. I'm having some trouble tracing the code in the few minutes I spent looking through each of these examples.


In the semver example, I think the responsible code is https://github.com/dtolnay/semver/blob/9a2084502fb595a1e016a9c72d333956664e0c48/src/identifier.rs#L241-L247

// Shift out the 1 previously placed into the most significant bit of the least
// significant byte. Shift in a low 0 bit to reconstruct the original 2-byte
// aligned pointer.
fn repr_to_ptr(repr: NonZeroU64) -> *const u8 {
    // `lea rax, [rdi + rdi]`
    (repr.get() << 1) as *const u8
}

This looks like an int-to-pointer cast. Or perhaps it even counts as an OOTA pointer? rust-lang/unsafe-code-guidelines#291 doesn't mention doing arithmetic on an integer before converting it back to a pointer.


I think the miniserde one is totally beyond me. I suspect that what's at play here is a self-referential struct, which is made to compile with this code:
https://github.com/dtolnay/miniserde/blob/17c036fdcf54d00fb1f06a53a1b8db75694205db/src/place.rs#L25-L39

#[macro_export]
macro_rules! make_place {
    ($name:ident) => {
        #[repr(C)]
        struct $name<__T> {
            out: $crate::__private::Option<__T>,
        }

        impl<__T> $name<__T> {
            fn new(out: &mut $crate::__private::Option<__T>) -> &mut Self {
                unsafe { &mut *{ out as *mut $crate::__private::Option<__T> as *mut $name<__T> } }
            }
        }
    };
}

But in any case, this miniserde problem is a great case study for how SB needs better diagnostics. What I'm confident in is that Miri detects the error far from the invalid-by-SB unsafe code.

@RalfJung
Copy link
Member

RalfJung commented Dec 16, 2021

This looks like an int-to-pointer cast.

Ah yes. That is very hard to support with tag-raw-pointers. This seems similar to #1866 (comment) (ferrilab/bitvec#135).

Or perhaps it even counts as an OOTA pointer? rust-lang/unsafe-code-guidelines#291 doesn't mention doing arithmetic on an integer before converting it back to a pointer.

Presumably the arithmetic here results in exactly the same integer value as what was returned by a prior ptr-to-int cast. The problem is more that casting an int to a ptr is inherently a rather cursed operation (for memory that is 'known' to the language -- so this statement excludes accessing non-Rust-managed memory at fixed addresses; that is fine as long as the aliasing rules are otherwise followed). The cast can often be avoided by using something like this, but it can require significant refactoring:

// Converts 'addr' to a pointer using the provenance of 'prov'.
fn int_to_ptr_with_provenance<T>(addr: usize, prov: *const T) -> *const T {
  // Nowhere in this function do we cast an integer to a pointer!
  let ptr = prov.cast::<u8>();
  ptr.wrapping_add(addr.wrapping_sub(ptr as usize)).cast()
}

In this case it looks like NonZeroU64 is used to hold data that might have provenance... I would usually recommend NonNull<()> instead but that would have a different size on 32bit systems so I doubt it will work here. Or alternatively a union of a pointer and an integer, but that would kill the niche. Not sure if we have a good option with current Rust here.

@dtolnay
Copy link
Member Author

dtolnay commented Dec 16, 2021

So do all of these involve self-referential structs or 'owning-ref'-style pointers?

miniserde and semver probably count as that. syn is maybe? The data structure is an acyclic singly linked list of T across multiple Box<[T]> allocations.

@camelid
Copy link
Member

camelid commented Dec 17, 2021

(What does OOTA stand for?)

@saethlin
Copy link
Member

(What does OOTA stand for?)

Out Of Thin Air: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1217r0.html

@saethlin
Copy link
Member

Just a minor update: It looks like in the meantime @dtolnay patched miniserde so that it is compatible with raw pointer tagging (though where exactly I'm not quite sure), and I've put up PRs to semver and syn which whether or not they meet maintainer approval, pass Miri without changing the underlying data structures or degrading performance.

So I guess whether or not you say

I am opening this issue not primarily because I want someone to come fix my broken code

I do indeed struggle to resist this kind of challenge over the long term :p

@saethlin
Copy link
Member

saethlin commented May 6, 2022

PRs to semver: dtolnay/semver#273 and syn: dtolnay/syn#1166 are now merged.

I think we could close this now?

@RalfJung
Copy link
Member

RalfJung commented May 6, 2022

Looks like it. Thanks a ton @saethlin :)

@RalfJung RalfJung closed this as completed May 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-aliasing Area: This affects the aliasing model (Stacked/Tree Borrows) C-question Category: Not necessarily a bug, but someone asking for support
Projects
None yet
Development

No branches or pull requests

4 participants