fix: Prevent undefined behaviour from malicious AsyncRead impl

[Marwes]

AsyncRead is safe to implement but can be implemented so that it
reports that it read more bytes than it actually did. poll_read_buf on
the other hand implicitly trusts that the returned length is actually
correct which makes it possible to advance the buffer past what has
actually been initialized.

An alternative fix could be to avoid the panic and instead advance by
n.min(b.len())

[DoumanAsh]

This probably should be debug assert.

Writing bad implemention is unlikely to be intentional

[udoprog]

We need it to be enforced at all times, since it needs to be upheld to prevent UB. Similar to bounds checking slices. Conversely though it might be optimized away for many cases!

[DoumanAsh]

no, we do not.
If we do not trust the contract, then this trait itself is UB and should be marked as unsafe

[udoprog]

It specifically needs to be checked here because it's being passed to advance_mut below, which itself is an unsafe fn similar to Vec::set_len. Consider if this using a slice instead:

let n = ready!(self.poll_read(cx, &mut buf))?;
let data = &buf[..n];

Accessing the data from the slice (&buf[..n]) would be similarly bounds checked, even if the implementation of poll_read was incorrect.

[udoprog]

Actually, the contract of BufMut::advance_mut does seem to prevent this scenario:

Panics

This function may panic if cnt > self.remaining_mut().

Implementer notes

It is recommended for implementations of advance_mut to panic if cnt > self.remaining_mut(). If the implementation does not panic, the call must behave as if cnt == self.remaining_mut().

A call with cnt == 0 should never panic and be a no-op.

@Marwes ?

[Marwes]

It won't panic and the Vec passed here will call reserve to expand to the correct length. But it will also follow that by calling set_len https://docs.rs/bytes/0.5.3/src/bytes/buf/buf_mut.rs.html#999 which then allows uninitialized bytes to be accessed.

Separately, the advance_mut implementation seems dubious. resize(cnt); set_len(len + cnt); will always result in the Vec exposing undefined values to safe callers.

Either panicking (as the doc suggests) or just trusting that the passed cnt is correct and not calling reserve seems better by either being safer or by avoiding the reserve overhead.

[udoprog]

This is a really interesting PR. On a similar note, is an implementation that incorrectly reports the length (without overflowing) to be considered sound?

It's fairly straight forward to reason about zero-initialized buffers, because even if the read underflows, the contents of the buffer is well-defined at all points. Or an unsafe implementation which uses an uninitialized buffer and then makes sure that the length is appropriately updated. But here we have a situation where safe code has a fair amount of control over the argument to Vec::set_len, but constrained to whatever implements BufMut.

[Marwes]

@DoumanAsh Reading uninitialized bytes from safe code without the assert! (which is why it can't be a debug_assert!).

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5adef81554ca3c3174739ddaa43fb763

[DoumanAsh]

@Marwes Defensive programming is all good, but your scenario implies that trait itself is broken.
And you only attempt to work-around by adding assert, which is likely to be useless in 99.9% implementations.
No one in sane mind will implement such bad reader intentionally, it is likely to be unintentional mistake.

I know there were like tons of useless convo about broken AsyncRead and Read traits, but I feel like this PR is just work-around for it.
This assert should be optional at least

[Marwes]

The trait isn't broken, but an implementation of it can be. Since neither the implementation of the trait, nor the caller of read_buf involves any unsafe code it should be impossible to invoke undefined behavior regardless of how bad or wrong the code otherwise is.

I am aware that this may not be the best way to fix this but fact of the matter is that it needs to be fixed in one way or another so I'd prefer suggestions for other, better fixes instead of sweeping it away.

The current fixes that I am aware of is:

• This PR

• AsyncRead could be an unsafe trait instead, which would then put the responsibility on the implementator in the same way as if unsafe fn prepare_uninitialized_buffer were explicitly implemented.

• BufMut could be changed to assert that advance_mut to never do a set_len that isn't larger than remaining_mut (which implies no reserve call). All other implementors of BufMut should also be audited if this is the case.

• read_buf could advance by n.min(b.len()) instead.

[udoprog]

@Marwes

BufMut could be changed to assert that advance_mut to never do a set_len that isn't larger than remaining_mut (which implies no reserve call). All other implementors of BufMut should also be audited if this is the case.

Since reading the panic note on the contract to advance_mut, I don't see how an UB because of an overflow is possible. In fact, the implementation for Vec<u8> does reserve (which doesn't seem to follow the contract, but I'll open a separate issue on that!).

Worst case it seems like advance_mut will either:

1. panic
2. silently truncate cnt to remaining_buf()
3. grow the underlying buffer to accommodate (undocumented? behavior of Vec<u8>).

This would however cause uninitialized arrays of bytes to be propagated through the application, which might be UB, but of this I'm not sure. It's certainly a bug though. But this could also happen if an implementer of AsyncRead returns a value greater than the number of bytes it's filled to the buffer (but doesn't overflow). See #2030 (comment)

So in my view, the question is if returning wholly or partially uninitialized arrays of u8 is UB or not? If yes, it seems to me as if poll_read must be marked unsafe, or an unsafe alternative provided.

[udoprog]

Opened tokio-rs/bytes#354

[udoprog]

I was thinking about how this problem generalizes to std types and uninitialized buffers, and found this internals thread which is very much relevant: https://internals.rust-lang.org/t/uninitialized-memory/1652/10

@carllerche commented a fair bit in it, so maybe they can comment on this issue as well?

[Marwes]

This would however cause uninitialized arrays of bytes to be propagated through the application, which might be UB, but of this I'm not sure. It's certainly a bug though. But this could also happen if an implementer of AsyncRead returns a value greater than the number of bytes it's filled to the buffer (but doesn't overflow). See #2030 (comment)

This isn't an issue because prepare_uninitialized_buffer is called before hand. So the entire, passed buffer is either fully initialized (if prepare_uninitialized_buffer is the default implementation) or the trait implementor has explicitly implemented it which puts the responsibility on that implementation instead (since unsafe had to be written to implement the method)

[udoprog]

This isn't an issue because prepare_uninitialized_buffer is called before hand. So the entire, passed buffer is either fully initialized (if prepare_uninitialized_buffer is the default implementation) or the trait implementor has explicitly implemented it which puts the responsibility on that implementation instead (since unsafe had to be written to implement the method)

Ah, that's right. Thanks for pointing that out!

[Marwes]

Any movement on this? Does not feel great to leave a soundness hole open, even if the trait is likely going to change.

[carllerche]

Sorry for the delay, this got lost in my inbox...