Add experimental [cfgs] section to decribe the expected cfgs

[Urgau]

This PR adds new section experimental [cfgs] to the manifest to describe the expected cfgs and their values. This change was proposed on the Cargo zulip thread without any objection, so I'm proposing it.

The new section is gated under cargo -Zcheck-cfg with cfgs (-Zcheck-cfg=cfgs).

The syntax is straight forward, a simple key/values^*. _{With a custom values sub-key for the rare occasion where the values are unknown (empty list != unknown). This could be removed if not desired.}

[cfgs]
"use_libc" = []                # means `--check-cfg=values('use_libc')`
"loom" = ["yes", "no", "auto"] # means `--check-cfg=values('feature', "yes", "no", "auto")`

[cfgs."unknow_values"] # means `--check-cfg=names('unknow_values')`
values = false         # aka values are unknown (so everything is accepted)

What does this PR try to resolve?

This PR tries to add the missing piece of the all "Checking conditional compilation at compile time", that is, statically defining for a package it's expected cfgs. With all the other -Zcheck-cfg options it was possible to enable checking of well known values, well known names, feature cfgs and add thoses with build script; but there was no way (except RUSTFLAGS) to statically set the expected cfgs so that anyone compiling the project could have the unexpected cfgs warnings.

Also by adding this section we also unblock the feature, especially the well know names, which without this section (and probably and edition) would be a breaking change if enable as there would be no way of setting the expected cfgs.

How should we test and review this PR?

This PR should be reviewed commit by commit and tested with the automated tests and examples.

Additional information

The implementation use the already existent check_cfg_args function that adds the --check-cfg args to the rustc/rustdoc invocation, so nothing new here.

The implementation also use #[serde(untagged)] to be able to dezerialize (and serialize?) the [cfgs] and [cfgs.*] sections into one struct. First time using it, so if you think it's not the right tool, let me know.

Questions

• Should there be any restrictions on the name (like disallowing feature) or on the values (like disallowing any value with ") ?
• Should cargo give the possibility of expressing "unknown" values ?

[rustbot]

r? @epage

(rustbot has picked a reviewer for you, use r? to override)

[est31]

When someone sees [cfgs] they might think it's a list of enabled cfg values. One alternative would be [expected-cfgs] but that can be misinterpreted as "these are the cfgs that the program needs to have set otherwise it doesn't compile".

I think [allowed-cfgs] would be best.

[Urgau]

When someone sees [cfgs] they might think it's a list of enabled cfg values. One alternative would be [expected-cfgs] but that can be misinterpreted as "these are the cfgs that the program needs to have set otherwise it doesn't compile".

I thought the same thing until I realized that the same problem is encountered by the [features] section, which like the [cfgs] section only describes what is expected, not what will be enabled. I also think it is better to be consistent with the [features].

Given that, I would prefer to stay with the simple name cfgs.

[est31]

@Urgau that analogy is a good argument that can be used in teaching 👍 . I rest my case.

[bors]

☔ The latest upstream changes (presumably #11793) made this pull request unmergeable. Please resolve the merge conflicts.

[ehuss]

Some initial comments:

• New syntax for Cargo.toml should be gated with cargo-features (see core/features.rs for more). The reason for that is if someone publishes a manifest with this syntax, and then in the future we change the meaning of this table, and then stabilize it, those stable releases will misinterpret the previously published manifest.

• What kind of considerations are there for workspaces? Should this support inheritance?

[Urgau]

* New syntax for `Cargo.toml` should be gated with `cargo-features` (see `core/features.rs` for more). The reason for that is if someone publishes a manifest with this syntax, and then in the future we change the meaning of this table, and then stabilize it, those stable releases will misinterpret the previously published manifest.

Will do it after the initial review, unless you what me to do it before.

* What kind of considerations are there for workspaces? Should this support inheritance?

I have no idea. At first glace I would say that it should have the same behavior that features has, and unless I'm missing something [features] are not a thing in workspaces.

But as said I don't really know, maybe we could wait and see if people want it and then implement it 🤷.

[est31]

I have no idea. At first glace I would say that it should have the same behavior that features has, and unless I'm missing something [features] are not a thing in workspaces.

I think [features] is usually very crate specific. It can happen that features get passed through a hierarchy of crates but if the chains get too long those cases are better served by not having passing and instead relying on the end user depending on the end crate in that chain and setting the features manually.

For cfgs use cases that apply to an entire workspace are more likely. For example, the compiler wants to allow #[cfg(bootstrap)]. Non compiler use cases exist probably too, although i cannot think of any.

The other question is about where the foo-bar to foo_bar translation should happen or if foo-bar = [] should just be disallowed.

[ehuss]

The cargo team discussed this a bit and had a fair bit of confusion and several questions:

• What specifically is the difference between this and using cargo:rustc-check-cfg in a build script? From my understanding, one issue is that using a build script is unable to specify valid cfg names for the build script itself. Am I understanding it correctly? Can you say more about the difference and why this would be separate?
• This seems like it could end up with a viral nature. For example, let's say I use tokio and want to occasionally use the tokio_unstable cfg. That seems like it would require me to add that to my own [cfgs] table to avoid warnings. Is that correct?
• Would it be possible to collect several different real-world use cases for situations where projects use arbitrary cfg values? It seems like several of the use cases are for working around limitations in the features system, and it seems like this could just be papering over those limitations, introducing a mechanism that we might later decide we don't want users to use. We'd like to have a clearer understanding of why people are using arbitrary cfg in the first place and if there are alternatives.

[Urgau]

What specifically is the difference between this and using cargo:rustc-check-cfg in a build script? From my understanding, one issue is that using a build script is unable to specify valid cfg names for the build script itself. Am I understanding it correctly? Can you say more about the difference and why this would be separate?

The main difference (at least for me) is convenience, it's way simpler to set the expected cfgs in the Cargo.toml than to have a build.rs with the correct build script invocation and knowing the required --check-cfg invocation. It also should allow checking for cfgs in a build.rs, because for checking them we need to be aware of them before building it (however I don't think it's currently done with this PR).

This seems like it could end up with a viral nature. For example, let's say I use tokio and want to occasionally use the tokio_unstable cfg. That seems like it would require me to add that to my own [cfgs] table to avoid warnings. Is that correct?

If you use RUSTFLAGS to set the cfg than yes, but the problem would appear without [cfgs], it could also appear with a build script or with -Zcheck-cfg=values/names.

Would it be possible to collect several different real-world use cases for situations where projects use arbitrary cfg values? It seems like several of the use cases are for working around limitations in the features system, and it seems like this could just be papering over those limitations, introducing a mechanism that we might later decide we don't want users to use. We'd like to have a clearer understanding of why people are using arbitrary cfg in the first place and if there are alternatives.

Sure, here are some examples:

• rust-lang/rust use bootstrap in it's entire codebase for compiler staging purpose
• tokio-rs/tokio use tokio_unstable because Cargo doesn't support feature that may break semver
• bytecodealliance/rustix use rustix_use_libc to be able to force the use of libc instead of raw syscall even in platform that supports it (but they also have the use-libc cargo feature)

[est31]

There is also the practice of having cfg's set by build scripts depending on the version of the rust compiler used. E.g. serde uses it.

you bring up good points though, @ehuss. Now I'm less convinced that a [cfgs] section is good :). On the other hand, the RFC for checking cfg's was meant to be opt-in. It would be sad that the only way to opt in would be by writing a build.rs.

[ehuss]

The Cargo team discussed this a bit, and I'm going to close this PR at this time. We feel like this feature needs more fundamental design work, and doing that via a PR doesn't work very well.

I would encourage you to work on hashing out the high-level design first before moving to an implementation. This feels like it is something that a full RFC would require. The original RFC left a lot of the Cargo side of the design unspecified, and that work should start with a design process similar in scope to an RFC. Not only addressing the motivation, but also writing several stories for how people use custom cfgs could help solidify if this is the right thing to do or not. Unfortunately the Cargo team doesn't have much bandwidth to help you with that at this time. You may consider reaching out on https://internals.rust-lang.org/ to start discussions to see if others could help contribute feedback on this.

In the short term, you may want to look at trying to stabilize cfg validation for built-in values. That would not require any effort from the user or Cargo. That could help with validating the values for things like cfgs like target_*, panic, etc.

I also encourage you to look at the possibility of just stabilizing validation of --cfg=feature="…" in the short term. I think that would be extremely high value without diving into full validation of all cfg keys.

In the medium term, we could look at stabilizing something that could work with build scripts that set custom cfgs. For example, I could imagine something like the way the openssl crate uses cargo:rustc-cfg which is not expected to be set by the user as a possible use-case that could be addressed (though it is not clear if that will work with someone setting a cfg for an unrelated crate via RUSTFLAGS).

As an initial set of feedback on this current approach, I have some concerns which seem to speak to the fundamental approach here:

• The viral nature where the [cfgs] block from some dependency doesn't help if a user sets those cfgs via RUSTFLAGS. For example, RUSTFLAGS="--cfg tokio_unstable" cargo build would result in warnings in my own crates since the --cfg is being passed to everything.
• This in essence is blessing a secondary system to Cargo's feature system. That seems like the wrong thing to do, or at least the wrong way to approach it. I understand users can already pass custom cfgs, and that is even expected in some cases, but to then move it front-and-center in Cargo.toml seems like it is bringing a low-level feature to the high-level without considering the interaction with the current feature system. If there are deficiencies in Cargo's features, the we should address those, not add a secondary system. For example, if there is a need for top-level-only features that limit how they are used. Or any of the other more advanced things like private/hidden features (private/hidden features #10882), unstable/nightly features (stable/unstable/nightly-only features #10881), features with custom values, etc.

Thank you for working on this! I know this is not the answer you want to hear. I definitely feel that validating cfgs will be useful. It just isn't clear to me at this time how full key validation can integrate with Cargo.

[Urgau]

Thank you for working on this! I know this is not the answer you want to hear. I definitely feel that validating cfgs will be useful. It just isn't clear to me at this time how full key validation can integrate with Cargo.

No worries, I my-self had some reservation after your comments.

As you suggested I'm going to focus my attention on getting well known values and as well as cfg(feature = "...") checking working. Thanks for your time.