Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default to no_std #2543

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Conversation

tcharding
Copy link
Member

@tcharding tcharding commented Mar 6, 2024

Default to no_std for hashes and bitcoin - all other crates are already done.

Turns out this one was a bit more involved that I expected, its a fair bit of messing around in tests but I think its worth it to give us more confidence in no_std.

Close: #2413

@github-actions github-actions bot added C-bitcoin PRs modifying the bitcoin crate C-hashes PRs modifying the hashes crate labels Mar 6, 2024
@tcharding
Copy link
Member Author

Strange errors showing up in CI on this and other PRs?

@apoelstra
Copy link
Member

This one looks real; it's related to imports of types which are affected by nostd.

@tcharding
Copy link
Member Author

No worries, thanks. Looks like I have some more work to do. Will come back to this.

@tcharding tcharding marked this pull request as draft March 7, 2024 01:30
@github-actions github-actions bot added the test label May 3, 2024
@coveralls
Copy link

coveralls commented May 3, 2024

Pull Request Test Coverage Report for Build 8946096346

Details

  • 19 of 20 (95.0%) changed or added relevant lines in 9 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.01%) to 83.174%

Changes Missing Coverage Covered Lines Changed/Added Lines %
hashes/src/hmac.rs 0 1 0.0%
Totals Coverage Status
Change from base Build 8939996688: 0.01%
Covered Lines: 19209
Relevant Lines: 23095

💛 - Coveralls

@tcharding tcharding force-pushed the 03-06-no-std branch 2 times, most recently from 59d831f to 26f6703 Compare May 9, 2024 04:06
@tcharding tcharding marked this pull request as ready for review May 9, 2024 05:38
@tcharding
Copy link
Member Author

Note to self, consider updating manifest catagories and keywords to include no-std same as in bitcoin-io.

@apoelstra
Copy link
Member

Hashes does not build with --no-default-features --features serde 5f38652

@tcharding
Copy link
Member Author

tcharding commented May 15, 2024

Interesting, that exposes a hole in CI also then. Will investigate, thanks. Tip your local CI bot for me, he is doing good work.

EDIT: oh, this is not a hole in CI, slipped through because we don't check every patch only the final PR state.

Either way is fine, just pick one and be uniform.
As we do for other crates; default to `no_std`.

There are currently two major problems with defaulting to `no_std`:

1. Many of the units tests use types/macros from `alloc`.
2. The "schemars" feature has an implicit dependency on "alloc", the
current code is buggy but we only ever test "schemars" with the "std"
feature enabled.

Solutions:

1. Just unconditionally import a bunch of `alloc` types.
2. Add a new feature "schemars" and change the dependency to be
"actual-schemars" then add all the feature gated imports.

Note also the usage of `use std::is_x86_feature_detected;`
As we do for other crates; default to `no_std`.

The major issues with this is that we have, up until now, assumed we
have `extern crate std` in tests. Most of this patch is fixing the
feature gating to use stuff from `extern crate alloc` and correctly
feature gating on "std" if required.

Note also use of `format!` in tests instead of `println!` because it is
available in `alloc`.
@tcharding
Copy link
Member Author

This is getting a bit silly, one has to wonder at if there is not a better way

#[cfg(test)]
mod tests {
    #[cfg(feature = "alloc")]
    #[allow(unused_imports)] // Less maintenance if we just import these.
    use crate::alloc::{format, string::ToString, vec, vec::Vec};
    #[cfg(any(feature = "alloc", feature = "serde"))]
    use crate::{sha256d, Hash as _};

@apoelstra
Copy link
Member

This is partially fallout from rust-lang/rust#121708 .. though I think it actually predates that issue.

We have a couple options. One is to just allow redundant imports on all our test modules. I'm not thrilled with this because imports do become stale. A better one would be (similar to what you've done) to cfg_attr allowing redundant imports unless all features are enabled. (But I'm uncertain how to specify "all features". Could be gross to do.)

@tcharding
Copy link
Member Author

tcharding commented May 15, 2024

What if we had a test prelude module (ie., gated on #[cfg(test)]) that imported all the alloc related rubbish and also all the individual hash modules, and we allow unused imports in it. Then in each unit test can just do use crate::test_prelude::*;

I believe that wildcard imports do not trigger redundant import warnings, which is nice. We'd have to be careful that for crates with a prelude that it and the test_prelude didn't get out of sync.

@tcharding
Copy link
Member Author

Could be gross to do.

"gross", what are you, five?

(Not sure if that joke translates to text, but I mean it in the most jovial of ribbing ways.)

@apoelstra
Copy link
Member

I'm not a huge fan of prelude modules, even for tests, because it means that you can't search for the location of symbols within the file that they appear in.

@tcharding
Copy link
Member Author

Yeah I've heard you say that before, maybe I'm missing something, but when do you ever need to search for alloc/core/std type e.g,. Vec?

@tcharding
Copy link
Member Author

tcharding commented May 15, 2024

Oh, I literally just suggested putting non-std types in a prelude :(

@tcharding
Copy link
Member Author

Just FTR this PR is good to merge, we are just having a discussion on the pain of imports for no_std crates.

@apoelstra
Copy link
Member

but when do you ever need to search for alloc/core/std type e.g,. Vec?

To understand what set of feature-gates are controlling their import logic, which is important when reviewing code that should work in a no-std context.

But yes, for non-std types the problem with wildcards is even clearer.

@tcharding
Copy link
Member Author

but when do you ever need to search for alloc/core/std type e.g,. Vec?

To understand what set of feature-gates are controlling their import logic, which is important when reviewing code that should work in a no-std context.

Interesting, so are you saying that when you review no-std code you are thinking "what are the feature gate requirements for this code"? And then you like to look at the imports to see if it matches your thoughts?

I always have treated prelude to mean "there are a set of feature gated imports that work with this code" and because its in one place it implies "and they are sane because we looked before and don't change them much". Where as when the feature gated imports are in each place I have to reason about them to check if they are sane.

@apoelstra
Copy link
Member

Interesting, so are you saying that when you review no-std code you are thinking "what are the feature gate requirements for this code"? And then you like to look at the imports to see if it matches your thoughts?

No, only when they break.

they are sane because we looked before and don't change them much

If only..

@tcharding
Copy link
Member Author

Needs rebase, this feature gating is boring as hell - leaving rebasing until after the one-ack thing comes in and we have some hope of merging this without more rebasing.

@tcharding tcharding marked this pull request as draft May 17, 2024 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bitcoin PRs modifying the bitcoin crate C-hashes PRs modifying the hashes crate test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Default to no_std in all crates
3 participants