Tracking Issue for strict_provenance #95228
Comments
Gankra
added
T-libs-api
FAQWhy Is Rust So Broken? This Clearly Isn't A Problem For C!It is absolutely a problem for C. Rust, C, C++, Swift, etc. are all fundamentally built on the same tools and principles once you start dropping down to the level of memory models (i.e. Rust literally just punts on atomics by saying "it's the C11 model" because that's the model for atomics). Compiler backends currently do not consistently model pointers in the face of things like "but pointers are just integers right"? Why Doesn't Rust Use C's SolutionFolks who work on C's semantics are still trying to solve this issue, with the leading solution being PNVI-ae-udi. It's an interesting and reasonable approach under the assumption of "we can't possibly get people to fix their code, and we can't completely change compiler backends". At a high-level, the proposal is to (in the abstract machine's semantics):
The strength of this model is that from a compiler's perspective this can mostly just be understood as "business as usual" because you can still let programmers do ~whatever and reason locally along the lines of: "I have kept perfect track of these pointers, and therefore know exactly how they are/aren't aliased. These other pointers over here have escaped or come from something I don't know about, so I will assume they all alias eachother and compile their accesses very conservatively." What they do need to change under this model is to admit that a ptr-to-int cast has a "side-effect" (exposing the allocation) and that optimizing it away is unsound, because then you will forget the pointer was exposed and do bad things. The weakness of this model is that essentially implies that an int-to-ptr cast has permission to access whatever memory you want (all "exposed" memory). This makes it very hard for dynamic checkers to be useful, because anytime ptr-int-ptr things happen, even implicitly/transiently, the checker has to throw up its hands and say "I guess you know what you're doing" and won't actually be able to help you catch bugs. For instance if you use-after-free, the checker cannot notice if you get "lucky" and start accessing a new allocation in the same place if the pointer was cast from an integer after the reallocation. This is Sad. It is perhaps an inevitable fate that Rust will adopt C's model, or at least have to interoperate with it, but it would be nice if we could do better given how much time and energy we put into having more rigorous and safe concepts for borrows and lifetimes! Rust is uniquely positioned to explore stricter semantics, and has established precedent for just saying "hey actually this idea we've had since time immemorial is busted, let's migrate everyone off of it so the language can make sense". I am not saying we are going to break the world right now, but we should explore how bad breaking the world is, and at worst, making code conform to strict provenance in more places will make our existing tools work more reliably, which is just Good. But CHERI Runs C Code Fine?Yes! Sort of. Pointer-integer shenanigans are not actually that common, and so most code actually already trivially has dynamic/strict provenance in both Rust and C. Most of the time people are just comparing addresses, checking alignment, or doing tagged-pointer shenanigans. These things are totally fine under strict provenance! The remaining places where people are actually committing pointer-integer crimes are handled by a hack that mostly works: defining intptr_t to just actually be a pointer still and having the compiler handle/codegen it as such. This is one of the genuine successes of C's model of "define a million different integer types that sure sound like they should be the same thing but get to be different if an implementation says they are". In particular, making this tolerable requires CHERI to say The intptr_t hack was explored for Rust but it doesn't work very well, because rust doesn't make a distinction between size_t and intptr_t. This meant every array size and index was 128-bit and handled by CHERI's more expensive pointer registers/instructions on the paranoid assumption that it could all be Secret Pointers We Need To Track. For Rust to properly support CHERI it needs to decouple the notions of size_t and intptr_t, which means we need everyone to be more clear on what they mean when they convert between pointers and integers. It would also be nice if Rust, the systems language that Really Cares About Memory-Safety was a first-class citizen on CHERI, the architecture that Really Cares About Memory-Safety. We are pursuing the same goals, and Rust's design is seemingly very friendly to CHERI... except pointer<->integer shenanigans make everything into a muddy mess. Isn't It A Big Deal That CHERI "Breaks" wrapping_offsetCHERI's pointer compression scheme falls over if you wrapping_offset too far out-of-bounds (like, kilobytes out of bounds), and will silently mark your pointer as "corrupt" (while still faithfully preserving the actual address). Once this happens, offsetting that pointer back in bounds and trying to load/store will fault. It's annoying but it's not really a problem. It's a system limitation, and if you run afoul of it you will get a deterministic fault. This is much the same as targetting Rust to some little embedded system, having to disable all of libstd, and then still crashing because you were too sloppy with your memory footprint. Or how random parts of std have to be cfg'd out when targetting WASM. Some code just isn't as portable as you'd like, because anything more exotic than x64 has quirky little limitations. Rust is not the intersection of all platform limitations, because that intersection is terrible. Also I should clarify something because it seems to have been lost to history:
Yes I know
I wrote those docs, so this is my fault, mea culpa. The fact that you "wanted" to use Your code isn't terrible for using wrapping_offset, I just should have made it more clear that it should be regarded as a Last Resort and not "the chill offset for everyone". 😿 Isn't This Model WAY Too Strict?Probably! My goal is not to define The One True Memory Model for Rust, but to define a painfully simple one that is easy for Rust programmers to understand and make their code conform to by default. The "idea" with strict-provenance is that it's so strict that any coherent model must surely be a weakening of it (i.e. says strictly more code is allowed). In this way, code that happens to conform to strict-provenance (which is most code, as CHERI has demonstrated!) is essentially guaranteed to be correct under any model and to not be miscompiled (barring compiler bugs). One can imagine ending up with this "tower of weakenings":
If your code works higher up the tower, it will definitely work against anything lower down the tower, and the bottom of the tower is the one that "matters", because that's the thing that actually compiles your code. It is frustrating as a programmer to know that there is this vague memory-model stuff going on, and that compilers are vaguely broken because they don't really have coherent models. By making it easier for code to conform to strict-provenance, we are making it more robust in the face of inconsistent and buggy semantics AND future-proofing that code against any possible "real" model. Mega Link PileThe Work On This API:
Prior Art For This API:
CHERI Resources: Provenance Resources:
Strict Provenance Zulip Threads:
|
|
The proposed lints in #95199 should be something a user messing with these APIs can opt into to quickly find sketchy places in their code. What's the "right" way to expose an unstable lint? Is it sufficient to make it allow and users can opt in with normal linting stuff, or do we also need a special feature/-Z to opt into the lint existing at all? |
This should probably be more like
I... hm. I think the allow-by-default lint is conservative enough? It shouldn't need a |
|
At least in the bootstrap, the compiler will complain if you
Also due to the "Opaque Function Pointers" / "Harvard Architecture" / "AVR is cursed" issue rust/library/core/src/ptr/mod.rs Line 1390 in 9280445 I think we want the lint broken up into parts:
I can't justify discouraging |
Hm... I think we can make a lint conditional on |
|
Overall I like this idea for Rust, but it seems incompatible for C interop. It's pretty common in C APIs to use an integer as a "user-data" field intended to store arbitrary values (primarily pointers). For example, the winapi I could imagine having some kind of built-in "pointer map" type, which behaves like a |
|
I feel like it's very plausible to define some sort of pointer-int union without messing with ABI for "this is a pointer, the API is lying" since in general, afaict, it's always sound to say something that is actually just an integer is a pointer "for fun" (as ptr::invalid shows) as long as you only deref/offset it when it's a Real Pointer. |
Yes indeed, pointers are a superset of (equally-sized) integers. |
But then you can't mechanically map C signatures to Rust, because you can't know whether an integer should be treated as a pointer or not. Today, basically everything you can do in C, you can also do in unsafe Rust, which makes it possible to call APIs designed to be called from C, but if Rust can't just "do what C does", then it makes that interop a lot harder. |
|
I mean, yes, but if you actually use the API and it expects you to pass it a dereferenceable pointer where it says the arg is an integer, then at that point you can go "ah, this API is lying" and do whatever needs to be done about that. Blindly charging forward, slamming into problems, and forcing ourselves to figure out what "do whatever needs to be done" should look like is the primary mission statement of this experiment. |
|
So, for that specific API, SetWindowLongPtrW, the official signature looks like this: LONG_PTR SetWindowLongPtrW(
[in] HWND hWnd,
[in] int nIndex,
[in] LONG_PTR dwNewLong
);where So in "real programs" you're expected to:
Also, while it's sometimes UB to declare the wrong foreign signature, that's because of cross-lang LTO, and we never normally compile user32, so we can just declare the wrong signature and type the function as: extern "system" {
pub fn SetWindowLongPtrW(hwnd: *mut c_void, index: c_int, new_long: *mut c_void) -> *mut c_void
}And now we "don't have to" perform pointer to int casting ourselves, it just silently happens during the foreign interfacing. All that said, when we get the pointer back from |
|
Could you elaborate on why provenance "must" be lost? If you're operating on a system that actually dynamically maintains provenance, the information must be maintained by the callee anyway or the OS literally doesn't function. If you're operating under a model where the rust abstract machine just needs to be self-consistent, then presumably having the API signature reflect "pointer goes in, pointer goes out" is sufficient? Like yes the compiler doesn't "know" where those pointers go or come from, but the compiler also doesn't "do" global analysis and therefore must be able to cope with calling a native rust function with a |
|
Well if there's no global analysis then that's fine, sure. |
|
Agreed with @Gankra.
|
It does not, it returns an integer, but presumably you are suggesting redefining the function such that the return type preserves provenance. |
|
Yes, @Lokathor suggested above to adjust the signature of |
Just as a note, this is in fact what the LLVM IR function signature looks like when building C/C++ with CHERI LLVM (https://cheri-compiler-explorer.cl.cam.ac.uk/z/KG5oqq).
|
|
@arichardson presumably that's because |
|
@Diggsey yes, see "A secondary goal" in the top comment and "But CHERI Runs C Code Fine?" in the FAQ. |
|
I think my concerns with C interop basically boil down to this: Rust might need a way to interoperate with the PNVI-ae-udi model (assuming that is what C goes with) and I'm not convinced that the boundary between Rust's provenance model and the C provenance model can lie precisely on the FFI boundary. I think there will always be cases where we need unsafe Rust to be able to "act like C" for that interop to be practical. If that concern turns out to be well-founded, it doesn't mean we can't still do better than C though. For example, we could use the same model, but have the "act of exposing a pointer" be an explicit compiler intrinsic intended for unsafe FFI code, rather than happening on all pointer-to-int casts. |
|
Yes it's probable there will need to be a way to say "I give up" and use a Ptr16/Ptr32/Ptr64/Ptr128 "integer type" that is exactly like how CHERI handles intptr_t. Solutions like this will be considered more seriously as we attempt to "migrate" more code to stricter semantics and run into limitations with the MVP. In terms of things like interoperating with PNVI-ae-udi at the level of cross-language LTO -- it is wholy premature to at all think about that. Like, 5-10 years premature, realistically. Having a proposed experimental memory model is in a completely different galaxy from actually emitting aliasing metadata into LLVM. |
|
If we don't consider cross-language LTO, then the FFI boundary is where such interop happens and I don't think we need to worry much about PNVI. Interactions all happen on the 'target machine' level, where most of the time there is no provenance (and when there is, like on CHERI, it is very explicit and propagates through integer and pointer types equally). If we do consider cross-language LTO, then really the interactions are described by a "shared abstract machine" that the optimizations happen on -- probably the LLVM IR Abstract Machine. It is anyone's guess how that one will account for PNVI, but given that the trade-offs are very different between surface languages and IRs (and given that LLVM IR does not enforce TBAA on all accesses) I doubt they will just copy PNVI. So it is rather futile IMO to try and prepare for this future. (And @Gankra wrote basically the same thing at the same time.^^) |
|
I think it would be a good idea to preemptively mark |
|
@ketsuban |
|
I'm not sure that's true, since const eval doesn't allow dereferencing pointers, so whether it is exposed or not shouldn't matter. It should just be able to treat |
|
I don't think |
|
It would pick from the set of exposed pointers...? I think the better question is what makes a pointer exposed, and we already basically have an answer to that ( |
|
The current main use of pointers in const is (i strongly suspect) to make a pointer to mmio. They're not accessed during const, but they're made and named during const. |
|
There is a set of exposed pointers for each constat separately. Does it pick from the one in the current const, or the union of all of these sets?
When does the choice happen, at compile time, at link time, or on program startup?
|
|
Sorry, wait, you're saying each const declaration has a separate provenance universe it works within? |
It doesn't matter, pick your favorite. I claim that pointer provenance is not observable at const time because you can't dereference the pointers anyway.
There is a complicated and to some extent TBD process for transferring the results from const time to runtime, but I imagine that it will involve exposing all allocations that could not be garbage collected and will end up in the final binary. |
That's the current behavior, yes. We don't really have a story for how provenance carries over from compile-time to run-time.
When you deref them at runtime, it matters which provenance we assign to them. |
True. But as I alluded to, I think that's an entirely separate question. I fully expect provenance to get completely scrambled and/or reinterpreted as part of the const time -> runtime transition, and so the fact that const pointers can be dereferenced at runtime doesn't necessarily imply that const time provenance is observable. Probably discussion about this should move to rust-lang/unsafe-code-guidelines#424. |
I think there is something missing here, because you can dereference raw pointers in |
As @Lokathor said, I'm making pointers to memory-mapped hardware registers. This comment advises |
Currently it does for aliasing but it's completely unclear to me what that means with from_exposed_addr. The AllocId still matters. The issue you link is specifically about aliasing restrictions, which is not the same as provenance in general.
It is still the case. It's just not at all clear what it means to do that at "const" time. Currently, at const time, |
|
Another reason for moving along with #98593 it seems. A |
We have to jump through a few hoops but it is possible for the callback to work only in terms of raw pointers and references without ever instantiating an actual `Box` or `Arc` and therefore having to worry about drop. In the absence of [strict provenance][] (nightly only unstable feature) we do still need the `Box` in order to make a thin pointer to the fat `dyn Tls13SecretCallbacks` which we need to access. (Aside: I think making `Session` generic over a `CB: Tls13SecretCallbacks` would avoid the box, as we do with `IOCB`, however for the keylogger debug facility we don't need to be quite so efficient and the generics get everywhere) However the `Box` should be part of `Self` so that the required lifetime is established. From that we can obtain a (thin) reference to the allocation within the `Box` which contains the `Arc<dyn...>` which contains a fat reference to the actual callback object. That thin reference from the `Box` can then be turned into a raw pointer (which a fat pointer cannot without strict provenance). The `&mut **self.secret_cb.as_mut().unwrap()` construct is tricky: - `self.secret_cb.as_mut().unwrap()` has type `&Box<Arc<dyn...>>`; - therefore `*...` is the `Box<Arc<dyn...>>` itself; - therefore `**...` is the allocation within the box i.e. `Arc<dyn...>>`; - finally `&**...` is a reference to that allocation i.e. the `&Arc<dyn...>` which we need. [strict provenance]: rust-lang/rust#95228
We have to jump through a few hoops but it is possible for the callback to work only in terms of raw pointers and references without ever instantiating an actual `Box` or `Arc` and therefore having to worry about drop. In the absence of [strict provenance][] (nightly only unstable feature) we do still need the `Box` in order to make a thin pointer to the fat `dyn Tls13SecretCallbacks` which we need to access. (Aside: I think making `Session` generic over a `CB: Tls13SecretCallbacks` would avoid the box, as we do with `IOCB`, however for the keylogger debug facility we don't need to be quite so efficient and the generics get everywhere) However the `Box` should be part of `Self` so that the required lifetime is established. From that we can obtain a (thin) reference to the allocation within the `Box` which contains the `Arc<dyn...>` which contains a fat reference to the actual callback object. That thin reference from the `Box` can then be turned into a raw pointer (which a fat pointer cannot without strict provenance). [strict provenance]: rust-lang/rust#95228
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
Summary: `1.76.0` release with fixes addressing the following: * Release notes ([link](https://releases.rs/docs/1.76.0/)) * Most notable is [#118054](rust-lang/rust#118054) manifesting as: ``` error: unused implementer of `futures::Future` that must be used --> fbcode/mlx/metalearner/housekeeper/housekeeper.rs:213:13 | 213 | self.ping_oncall(&oncall, usecases); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: futures do nothing unless you `.await` or poll them = note: requested on the command line with `-D unused-must-use` ``` * Changes in `search_index.js` spec for `rustdoc` ([link](https://github.com/rust-lang/rust/pull/118910/files#diff-3ac57789ddcd2856a3b4f0c444f2813315179bdbe55bb945fe64fcb27b53fee5L491)) * Split of `#![feature(exposed_provenance)]` ([link](rust-lang/rust#118487)) from [#95228](rust-lang/rust#95228) * `buck2` OSS toolchain bump to `nightly-2023-12-11` just before [#11878](rust-lang/rust-clippy#11878) and a bunch of other clippy lint renames. Reviewed By: dtolnay Differential Revision: D53776867 fbshipit-source-id: 78db83d8cdd6b0abae2b94ed1075e67b501fcd73
and others
Feature gate:
#![feature(strict_provenance)]read the docs
get the stable polyfill
subtasks
This is a tracking issue for the
strict_provenancefeature. This is a standard library feature that governs the following APIs:pointer::addrpointer::with_addrpointer::map_addrcore::ptr::invalidcore::ptr::invalid_mutThis is an unofficial experiment to see How Bad it would be if Rust had extremely strict pointer provenance rules that require you to always dynamically preserve provenance information. Which is to say if you ever want to treat something as a Real Pointer that can be Offset and Dereferenced, there must be an unbroken chain of custody from that pointer to the original allocation you are trying to access using only pointer->pointer operations. If at any point you turn a pointer into an integer, that integer cannot be turned back into a pointer. This includes
usize as ptr,transmute, type punning with raw pointer reads/writes, whatever. Just assume the memory "knows" it contains a pointer and that writing to it as a non-pointer makes it forget (because this is quite literally true on CHERI and miri, which are immediate beneficiaries of doing this).A secondary goal of this project is to try to disambiguate the many meanings of
ptr as usize, in the hopes that it might make it plausible/tolerable to allowusizeto be redefined to be an address-sized integer instead of a pointer-sized integer. This would allow for Rust to more natively support platforms wheresizeof(size_t) < sizeof(intptr_t), and effectively redefineusizefromintptr_ttosize_t/ptrdiff_t/ptraddr_t(it would still generally conflate those concepts, absent a motivation to do otherwise). To the best of my knowledge this would not have a practical effect on any currently supported platforms, and just allow for more platforms to be supported (certainly true for our tier 1 platforms).A tertiary goal of this project is to more clearly answer the question "hey what's the deal with Rust on architectures that are pretty harvard-y like AVR and WASM (platforms which treat function pointers and data pointers non-uniformly)". There is... weirdness in the language because it's difficult to talk about "some" function pointer generically/opaquely and that encourages you to turn them into data pointers and then maybe that does Wrong Things.
The mission statement of this experiment is: assume it will and must work, try to make code conform to it, smash face-first into really nasty problems that need special consideration, and try to actually figure out how to handle those situations. We want the evil shit you do with pointers to work but the current situation leads to incredibly broken results, so something has to give.
Public API
This design is roughly based on the article Rust's Unsafe Pointer Types Need An Overhaul, which is itself based on the APIs that CHERI exposes for dynamically maintaining provenance information even under Fun Bit Tricks.
The core piece that makes this at all plausible is
pointer::with_addr(self, usize) -> Selfwhich dynamically re-establishes the provenance chain of custody. Everything else introduced is sugar or alternatives toascasts that better express intent.More APIs may be introduced as we explore the feature space.
Steps / History
Unresolved Questions
How Bad Is This?
How Good Is This?
What's Problematic (And Should Work)?
volatileaccess#[repr(transparent)] OpaqueFnPtr(fn() -> ())type in std, need a way to talk about e.g. dlopen.with something like llvm's proposed byte type
What's Problematic (And Might Be Impossible)?
APIs We Want To Add/Change?
.is_aligned(),.is_aligned_to(usize)?exists_zst(usize)?.addr()should arguably work on a DST, if you use.addr()you are ostensibly saying "I know this doesn't roundtrip".with_tag(TAG)?expose_addr/from_exposed_addrare slightly unfortunate names since it's not the address that gets exposed, it's the provenance. What would be better names? Please discuss on Zulip.addris the short and easy name for the operation that programmers likely expect less. (Many will expectexpose_addrsemantics.) Maybe it should have a different name. But which name?The text was updated successfully, but these errors were encountered: