Consider feature dependencies in test matrix #52

demmerichs · 2023-09-18T15:26:56Z

These two small commits fix #25 in that transitive, or dependent features are considered during the construction of the test matrix. It also closes #43.

Commit message:
Even though cargo activates dependent features automatically, we can reduce
the amount of feature sets when already resolving these dependencies during creation of all feature sets.
We decide to go for the more verbose option, selecting [A, B] over [B] when A is a feature dependency of B.
This allows an easier implementation, as dependency chains C -> B -> A can be verified one by one, and it takes other rules like
skip_feature_sets correctly into account without modification. (see __nightly example from #25)

BREAKING CHANGE: As previously feature sets [B], [A,B] were included separatly, now there will only be [A,B] run if A is a dependency of B. The testcase [B] is redundant, as it activates feature A automatically.

…lving skip_feature_set conflicts This has no change in behavior, as cargo_metadata guards against features appearing in both always_include and skip_feature_sets.

…e selected feature sets Fixes frewsxcv#25. Fixes frewsxcv#43. Even though cargo activates dependent features automatically, we can reduce the amount of feature sets when already resolving these dependencies during creation of all feature sets. We decide to go for the more verbose option, selecting [A, B] over [B] when A is a feature dependency of B. This allows an easier implementation, as dependency chains C -> B -> A can be verified one by one, and it takes other rules like skip_feature_sets correctly into account without modification. BREAKING CHANGE: As previously feature sets [B], [A,B] were included separatly, now there will only be [A,B] run if A is a dependency of B. The testcase [B] is redundant, as it activates feature A automatically.

frewsxcv · 2023-09-22T15:41:58Z

I'm having trouble understanding this change. Can you share a minimal Cargo.toml file which demonstrates how the behavior changes before and after this PR?

demmerichs · 2023-09-22T16:07:16Z

The first commit should change nothing, it was just a refactoring making the second commit easier.

The second commit performs the actual fix. All test cases in settings.rs were adapted in that commit to reflect the change in behavior. For all test cases, the default Cargo.toml used can be found at the end of settings.rs, but for reference:

        [package]
        name = "testdummy"
        version = "0.1.0"

        [features]
        A = []
        B = ["A"]
        C = ["dep:optDepC"]

        [dependencies]
        fixDepA = {path = "fixDepA"}
        oDepB = {path = "optDepB", package = "optDepB", optional = true}
        optDepC = {path = "optDepC", optional = true}

Feature C has just a crate dependency, no behavior change here, but feature B has a feature dependency on feature A. This means, when running the crate only with feature B enabled, cargo will automatically/implicitly activate feature A as well. If we now also run the feature combination A B, the behavior will be exactly the same as previously, where only B was explicitly enabled. So we can drop one of the two feature sets (and consequently any mix with additional features, e.g. one of B C and A B C) in order to reduce the amount of combinations run.

However, this is not only an optimization, but in conjunction with skip_feature_sets a true fix. Imagine, skip_feature_sets is set to skip combinations containing both feature A and C. Without this fix, the combination B C would run, even though feature A gets activated implicitly and now clashes potentially with the activated feature C. This is also the reason why I decided to drop all feature combinations where implicit feature activation occurs, as with explicit feature activation, all other rule verifications specified by user settings work as intended without any change.

Manishearth · 2023-10-02T20:18:20Z

Please add code comments so it's obvious what the code is doing and why

Manishearth · 2023-10-02T20:21:07Z

src/features_finder.rs

+                    .unwrap_or(&FeatureList::default())
+                    .iter()
+                {
+                    if dep.strip_prefix("dep:").is_some() {


note that this won't work for old-style implicit dependency features

but we haven't been careful about that in this tool anyway

The same mechanism is used in line 22. So I would leave this as is, and an issue could be opened discussing this further. I have no knowledge about the old-style you are referring to, so I personally would not open that issue ;)

Manishearth · 2023-10-02T20:23:51Z

src/features_finder.rs

@@ -87,6 +87,21 @@ pub fn fetch_feature_sets(package: &crate::cargo_metadata::Package) -> Vec<Featu
                // skip_feature_set matches: do not add it to feature_sets
                continue 'outer;
            }
+            for feat in feature_set.clone() {


we shouldn't have to clone here, we're not mutating. borrow it

Manishearth · 2023-10-02T20:26:08Z

src/features_finder.rs

+                        continue;
+                    }
+                    if !feature_set.contains(&dep) {
+                        continue 'outer;


thought: it would be nice if these skipped feature combos were still included in the output at the bottom, with an explanation of which run they are equivalent to. unsure if that's going to be really annoying to do properly, could be fine with "skipped covered feature set [...] due to dependency from .. to .."

Looking at my follow-up work regarding the rule abstraction, those dependencies would be encoded with the same rule mechanism as other selection rules, e.g. skip_feature_sets, denylists etc.
For this reason, I would not want to build this into this for-loop, but maybe externally creating a list of skipped feature sets because of dependencies. This could even be generalized, to print for every skipped feature set one or even all rules, that result in the set skipped. But I also think that in normal use cases, this output is unwanted.

What you are probably thinking of is helping users understand all the rules applied to the feature set selection and help "debugging" those rules. Probably, this workflow is started by recognizing a specific feature set is not run while the dev does not understand why it is skipped. So the dev wants to "ask" why a certain feature set was skipped. We could provide a CLI flag, taking a feature set as argument value, and we print all the conflicting rules to the screen, if it is skipped.

I think this would be an easy and nice addition to the tool, but I would like to implement it on top of the rule abstraction, because as of now, this would be quite tedious handling every "rule" case separately.

What do you think @Manishearth?

Yes, that's fine, also this was a "thought"; I wasn't requiring this of this PR.

I'm also not sure we should print every skipped set.

demmerichs added 2 commits September 18, 2023 17:14

refactor(features_finder): append always_include_features before reso…

98a8f35

…lving skip_feature_set conflicts This has no change in behavior, as cargo_metadata guards against features appearing in both always_include and skip_feature_sets.

demmerichs mentioned this pull request Sep 29, 2023

Integration tests, refactoring, and powerful feature rule expressions #50

Open

Manishearth reviewed Oct 2, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consider feature dependencies in test matrix #52

Consider feature dependencies in test matrix #52

demmerichs commented Sep 18, 2023

frewsxcv commented Sep 22, 2023

demmerichs commented Sep 22, 2023 •

edited

Manishearth commented Oct 2, 2023

Manishearth Oct 2, 2023

demmerichs Oct 2, 2023

Manishearth Oct 2, 2023 •

edited

Manishearth Oct 2, 2023

demmerichs Oct 2, 2023 •

edited

Manishearth Oct 2, 2023

Consider feature dependencies in test matrix #52

Are you sure you want to change the base?

Consider feature dependencies in test matrix #52

Conversation

demmerichs commented Sep 18, 2023

frewsxcv commented Sep 22, 2023

demmerichs commented Sep 22, 2023 • edited

Manishearth commented Oct 2, 2023

Manishearth Oct 2, 2023

Choose a reason for hiding this comment

demmerichs Oct 2, 2023

Choose a reason for hiding this comment

Manishearth Oct 2, 2023 • edited

Choose a reason for hiding this comment

Manishearth Oct 2, 2023

Choose a reason for hiding this comment

demmerichs Oct 2, 2023 • edited

Choose a reason for hiding this comment

Manishearth Oct 2, 2023

Choose a reason for hiding this comment

demmerichs commented Sep 22, 2023 •

edited

Manishearth Oct 2, 2023 •

edited

demmerichs Oct 2, 2023 •

edited