Conditionally enable functionality with feature flags #1498

asomers · 2021-08-22T17:26:12Z

This PR satisfies a long-running request: to slim down the Nix crate by conditionally enabling certain functionality. By default everything is enabled, but consumers can reduce that by selecting a subset of features.

Using features reduces build time and size for consumer crates. By default all features are enabled.

asomers · 2021-10-09T17:22:49Z

@rtzoeller @vdagonneau would either of you like to review this PR?

rtzoeller · 2021-10-09T18:40:14Z

@asomers I should be able to look at this PR this weekend!

rtzoeller

Running cargo doc --no-default-features revealed some items which were potentially missed when adding things to features.

nix::fcntl::SealFlag, for instance, seems like it should be restricted to the fs feature.

rtzoeller · 2021-10-09T18:47:58Z

CHANGELOG.md

@@ -6,6 +6,9 @@ This project adheres to [Semantic Versioning](https://semver.org/).
 ## [0.23.0] - 2021-09-28
 ### Added

+- Added fine-grained features flags.  Most Nix functionality can now be
+  conditionally enabled.  By default, all features are enabled.
+  (#[TODO](https://github.com/nix-rust/nix/pull/TODO))


Don't forget to update this link now that the pull request is enabled. It looks like this is also in the 0.23.0 release section, and needs to be moved.

rtzoeller · 2021-10-09T18:51:34Z

Cargo.toml

+  "acct", "aio", "dir", "env", "event", "features", "fs",
+  "hostname",
+  "inotify", "ioctl", "kmod", "mman", "mount", "mqueue",
+  "net", "personality", "process", "poll", "pthread", "ptrace", "quota",


The p's and q's aren't quite alphabetical, either in the default list or the list of feature dependencies below.

rtzoeller · 2021-10-09T18:53:38Z

src/lib.rs

+//! * `net` - Networking-related functionality
+//! * `personality` - Set the process execution domain
+//! * `process` - Stuff relating to running processes
+//! * `poll` - APIs like `poll` and `select`


Not quite alphabetical.

rtzoeller · 2021-10-09T19:04:49Z

src/fcntl.rs

@@ -165,6 +172,9 @@ libc_bitflags!(
    }
 );

+feature!{


I know nix doesn't use rustfmt (but maybe someday that can change!), but I think it's worth pointing out that rustfmt wants there to be a space between the ! and { in all of these feature macros. Might be worth a trivial bulk find and replace, since we're already adding noise to all these files' diffs.

Suggested change

feature!{

feature! {

rtzoeller · 2021-10-09T19:23:43Z

.cirrus.yml

@@ -20,6 +20,8 @@ build: &BUILD
    - $TOOL +$TOOLCHAIN $BUILD $ZFLAGS --target $TARGET --all-targets
    - $TOOL +$TOOLCHAIN doc $ZFLAGS --no-deps --target $TARGET
    - $TOOL +$TOOLCHAIN clippy $ZFLAGS --target $TARGET -- $CLIPPYFLAGS
+    - if [ -z "$NOHACK" ]; then $TOOL install cargo-hack; fi
+    - if [ -z "$NOHACK" ]; then $TOOL hack $ZFLAGS check --target $TARGET --each-feature; fi


There are enough features that it's not feasible to run cargo hack check --feature-powerset, but it may be useful to run it with limited depth.

$ cargo hack check --feature-powerset --depth 2

I found one issue (and possibly more, but I can't seem to get cargo hack check to not fail fast) by setting the depth to 2.

The following command fails:

$ cargo check --no-default-features --features mount,socket

Good catch. This became an error only after PR #1546, though it was probably wrong anyway. I'm not sure why I put that part about uio in there.

Running cargo doc --no-default-features revealed some items which were potentially missed when adding things to features.

nix::fcntl::SealFlag, for instance, seems like it should be restricted to the fs feature.

Good catch. I only checked the no-features docs on FreeBSD, where SealFlag doesn't exist. What about unistd::pipe? It doesn't seem essential, like read and write, but it also doesn't seem related to any other feature and I didn't want to create a new feature just for it.

Agreed it doesn't make sense to have a dedicated feature for it.

Given the options of sticking it in a feature now and potentially moving it to a different feature later (because we aren't happy with the location), or leaving it unconditionally present for now and potentially moving it to a feature later, I prefer the second option. Both would be breaking changes, but the second feels less confusing to me.

I think you should leave it as essential for now.

asomers · 2021-10-09T23:01:06Z

I think I addressed all of your comments. HOWEVER, rustdoc's behavior just changed, which will require revisiting every part of this PR. I'm working on that now.
rust-lang/rust#89596

rtzoeller · 2021-10-12T01:10:32Z

One other note: if you've considered updating the codebase to use rustfmt/cargo fmt, doing it immediately following the submission of this PR might be a good idea. This change will likely cause most existing PRs to require resolving conflicts anyways.

asomers · 2021-10-15T23:17:57Z

So I played with this some more, and I've come to the conclusion that the new rustdoc feature isn't good enough to be useful. While it's a step in the right direction, it just isn't good enough, so we'll basically have to code as if it weren't there. Worse, we'll have to carefully workaround the new feature so the docs don't become confusing. The problems are that:

Like other attributes, it doesn't work on macro invocations. So we'll have to explicitly set the #[doc(cfg(...)] on every libc_bitflags! and sockopt_impl!.
Putting the #[cfg(target_os=...)] restrictions in the documentation is handy at first glance, but problematic because:
- items that aren't built on the platform for which the docs were built won't show up in the docs. For example, if you build the docs on FreeBSD, you won't see that there's a kmod module only available for Linux.
- Using #[cfg(any(docsrs, target_os=...))] is not a solution, because too many items will simply throw errors when generating the documentation that way. For example, every enum value in a libc_enum! invocation and every const in a libc_bitflags! invocation.
- Some items exist for multple OSes, but need to be documented differently. For example, the mount module. There's no possible way to accurate document such modules except by building the docs for multiple platforms.
Annotating the docs with a bunch of OS notes gives readers a false sense of security. For the reasons noted above, those annotations will be incomplete, which might deter users from checking the docs built for their platform. So I think we should carefully remove all of those annotations from the docs.

This should be merged in a separate PR

The reasons not to use it are: * Items that aren't built on the platform for which the docs were built won't show up in the docs. For example, if you build the docs on FreeBSD, you won't see that there's a kmod module only available for Linux. * Some items are built with different signatures on different OSes. A "target_os" annotation won't tell the reader that. * Using #[cfg(any(docsrs, target_os=...))] is not a solution, because too many items will simply throw errors when generating the documentation that way. For example, every enum value in a libc_enum! invocation and every const in a libc_bitflags! invocation. * Annotating the docs with a bunch of OS notes gives readers a false sense of security. For the reasons noted above, those annotations will be incomplete, which might deter users from checking the docs built for their platform. So I think we should carefully remove all of those annotations from the docs. However, I'm leaving in place annotations about target_env and target_arch, because we don't build docs separately for all of those combinations.

rtzoeller · 2021-12-19T05:16:47Z

@asomers I think we should consider whether or not we expect tests with just some features enabled to work.

E.g. should cargo test --no-default-features --features fs build successfully, and run just the fs-related tests?

I'm also fine with saying "ideally yes, but not in this PR".

asomers · 2021-12-21T01:25:55Z

@asomers I think we should consider whether or not we expect tests with just some features enabled to work.

E.g. should cargo test --no-default-features --features fs build successfully, and run just the fs-related tests?

I'm also fine with saying "ideally yes, but not in this PR".

It would be nice. OTOH it would clutter up the test files with a lot of #[cfg()] directives, which is why I didn't do it originally. I don't think it's necessary, because I can't see any way that disabling features could cause the tests to fail, as long as they compile. Can you?

asomers mentioned this pull request Aug 22, 2021

Add cargo features #1197

Closed

asomers force-pushed the use-features branch 2 times, most recently from f76c43a to d17c09c Compare October 4, 2021 04:08

feature-gate most Nix functions

bf93bc0

Using features reduces build time and size for consumer crates. By default all features are enabled.

asomers force-pushed the use-features branch from 6a800c0 to bf93bc0 Compare October 9, 2021 17:17

rtzoeller reviewed Oct 9, 2021

View reviewed changes

asomers added 5 commits October 9, 2021 16:40

fix CHANGELOG entry

37fed26

Alphabetize features list

e11256f

Restrict SealFlag to feature(fs)

10a160f

Don't build from_libc_sockaddr in feautre(uio)

9463da3

style

d1a2bcd

asomers mentioned this pull request Oct 16, 2021

Correct definition of MADV_SOFT_OFFLINE on ppc #1566

Merged

rtzoeller mentioned this pull request Oct 16, 2021

Recent versions of Android support EPOLLEXCLUSIVE #1567

Merged

asomers added this to the 0.24.0 milestone Oct 16, 2021

asomers added 2 commits October 16, 2021 19:35

DO NOT MERGE fix a warning on the latest nightly

261fcb2

This should be merged in a separate PR

asomers mentioned this pull request Nov 17, 2021

MacOS: Undefined symbols for architecture x86_64 #1588

Open

rivy mentioned this pull request Nov 19, 2021

Fix/enhance CICD uutils/coreutils#2750

Merged

rtzoeller mentioned this pull request Nov 28, 2021

Avoid lock poisoning by using parking_lot #1599

Merged

asomers mentioned this pull request Dec 21, 2021

feature-gate most Nix functions #1611

Merged

asomers closed this Dec 21, 2021

This was referenced Aug 8, 2023

Do not enable any features by default smalis-msft/nix#1

Closed

Do not enable any features by default #2091

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Conditionally enable functionality with feature flags #1498

Conditionally enable functionality with feature flags #1498

asomers commented Aug 22, 2021

asomers commented Oct 9, 2021

rtzoeller commented Oct 9, 2021

rtzoeller left a comment

rtzoeller Oct 9, 2021

rtzoeller Oct 9, 2021

rtzoeller Oct 9, 2021

rtzoeller Oct 9, 2021

rtzoeller Oct 9, 2021

asomers Oct 9, 2021

rtzoeller Oct 10, 2021

asomers commented Oct 9, 2021

rtzoeller commented Oct 12, 2021

asomers commented Oct 15, 2021

rtzoeller commented Dec 19, 2021

asomers commented Dec 21, 2021

Conditionally enable functionality with feature flags #1498

Conditionally enable functionality with feature flags #1498

Conversation

asomers commented Aug 22, 2021

asomers commented Oct 9, 2021

rtzoeller commented Oct 9, 2021

rtzoeller left a comment

Choose a reason for hiding this comment

rtzoeller Oct 9, 2021

Choose a reason for hiding this comment

rtzoeller Oct 9, 2021

Choose a reason for hiding this comment

rtzoeller Oct 9, 2021

Choose a reason for hiding this comment

rtzoeller Oct 9, 2021

Choose a reason for hiding this comment

rtzoeller Oct 9, 2021

Choose a reason for hiding this comment

asomers Oct 9, 2021

Choose a reason for hiding this comment

rtzoeller Oct 10, 2021

Choose a reason for hiding this comment

asomers commented Oct 9, 2021

rtzoeller commented Oct 12, 2021

asomers commented Oct 15, 2021

rtzoeller commented Dec 19, 2021

asomers commented Dec 21, 2021