*: Remove default features from all crates #2918

thomaseizinger · 2022-09-20T03:54:09Z

Description

This patch-set removes our use of default features. We also introduce a new full feature that activates all features.

Links to any relevant issues

Open Questions

Change checklist

I have performed a self-review of my own code
~~I have made corresponding changes to the documentation~~
I have added tests that prove my fix is effective or that my feature works
A changelog entry has been made in the appropriate crates

This makes search and replace operations work better.

This crate does not have any default features.

Adds a CI check that ensures we specify the correct features for each example.

This gets rid of the cool but complicated CI check.

mxinden

Thanks for tackling this.

As mentioned elsewhere, I am in favor of this change.

Given that this is a large breaking change for users, is there anyone objecting to this change?

CHANGELOG.md

mxinden · 2022-09-21T13:32:30Z

core/Cargo.toml

@@ -46,7 +46,7 @@ ring = { version = "0.16.9", features = ["alloc", "std"], default-features = fal
 async-std = { version = "1.6.2", features = ["attributes"] }
 base64 = "0.13.0"
 criterion = "0.4"
-libp2p = { path = "../", default-features = false, features = ["mplex", "noise"] }
+libp2p = { path = "..", features = ["full"] }


Say I want to run the libp2p-core unit tests locally, wouldn't it be nicer to only compile the libp2p features that I need? Is this use-case worth the additional effort?

It is a pain though to keep track of the features you need to activate :/

Plus, with all crates requesting different features, I actually think we have to compile things multiple times to satisfy each combination. If the convention is to enable full everywhere, we only need to compile every crate once to and the compiled artifact can be used for every crate's tests¹.

I think the convenience of just saying "full" is worth it. Locally you have caches of everything so you will most likely just pay in linker time and if you modify your code between test executions, you gotta relink the test binary anyway so I am not sure it will matter in practice.

Can we try it out and if it negatively affects anything, figure out how to make it better?

Footnotes

This is an unproven hypothesis. ↩

Thanks for elaborating. Using full sounds good to me.

I changed my mind. Esp. working on libp2p-core is quite painful now because every change there invalidates libp2p-core artifact which causes a re-compile of every other crate.

👍 mind proposing a patch @thomaseizinger?

So I think the best way to resolve this is to actually stop the circular dependency. This will allow for proper caching of the build artifacts. It will also solve the issue I am currently facing with release-please (#2902).

Co-authored-by: Max Inden <mail@max-inden.de>

thomaseizinger · 2022-09-22T09:08:56Z

Given that this is a large breaking change for users, is there anyone objecting to this change?

Is it that large a change? Activate a couple of feature flags and your back in. No functionality is changing!

…p2p into 2173-no-default-features

thomaseizinger · 2022-09-23T04:06:29Z

Given that this is a large breaking change for users, is there anyone objecting to this change?

Is it that large a change? Activate a couple of feature flags and your back in. No functionality is changing!

I remember tokio moving to this convention taking me a couple of h. I am not saying it is hard. Just saying it is tedious. Never the less I am in favor of this change.

@thomaseizinger do you want this to be included in v0.49.0 (#2931)?

That would be great, yeah!

dignifiedquire · 2022-09-23T09:55:11Z

I would be great to have a short section "How to upgrade" for the release, so folks now how to adjust their feature usage.

thomaseizinger · 2022-09-23T10:16:05Z

I would be great to have a short section "How to upgrade" for the release, so folks now how to adjust their feature usage.

Is the first sentence in the changelog entry not good enough? https://github.com/libp2p/rust-libp2p/blob/2173-no-default-features/CHANGELOG.md#0490---unreleased

dignifiedquire · 2022-09-23T11:30:53Z

Is the first sentence in the changelog entry not good enough?

I somehow missed that when looking through the diff. LGTM 👍

mxinden · 2022-09-27T14:37:49Z

@thomaseizinger would you mind resolving the merge conflicts here? No rush, just a ping.

thomaseizinger · 2022-09-28T00:24:46Z

@thomaseizinger would you mind resolving the merge conflicts here? No rush, just a ping.

Sure! I just didn't realize there are any :/
I opened #2948.

thomaseizinger · 2022-09-28T00:35:06Z

@mxinden Resolved.

thomaseizinger · 2022-09-29T22:44:21Z

Wooooo! This is so exciting!

See libp2p#2918.

umgefahren · 2022-10-04T08:26:43Z

As long as #2950 is still open, this could result in a very confusing experience for a beginner. We should definitely fix that.

Circular dependencies are problematic in several ways: - They result in cognitive overhead for developers, in trying to figure out what depends on what. - They present `cargo` with limits in what order the crates can be compiled in. - They invalidate build caches unnecessarily thus forcing `cargo` to rebuild certain crates. - They cause problems with tooling such as `release-please`. To actually break the circular dependencies, this patch inlines the uses of `development_transport` in the examples and tests for all sub-crates. This is only meant to be a short-term fix until #3111 and #2888 are fixed. To ensure we don't accidentally reintroduce this dependency, we add a basic CI that queries `cargo metadata` using `jq`. Resolves #3053. Fixes #3223. Related: #2918 (comment) Related: googleapis/release-please#1662

Remove default features. You need to enable required features explicitly now. As a quick workaround, you may want to use the new `full` feature which activates all features.

Circular dependencies are problematic in several ways: - They result in cognitive overhead for developers, in trying to figure out what depends on what. - They present `cargo` with limits in what order the crates can be compiled in. - They invalidate build caches unnecessarily thus forcing `cargo` to rebuild certain crates. - They cause problems with tooling such as `release-please`. To actually break the circular dependencies, this patch inlines the uses of `development_transport` in the examples and tests for all sub-crates. This is only meant to be a short-term fix until libp2p#3111 and libp2p#2888 are fixed. To ensure we don't accidentally reintroduce this dependency, we add a basic CI that queries `cargo metadata` using `jq`. Resolves libp2p#3053. Fixes libp2p#3223. Related: libp2p#2918 (comment) Related: googleapis/release-please#1662

thomaseizinger added 15 commits September 20, 2022 11:20

Expose ecdsa feature in top-level libp2p crate

0f59322

Remove default feature from libp2p-core

9d59408

Always have the version first

16a4692

This makes search and replace operations work better.

Bump versions and add changelog entries

ce0708a

Remove default feature from libp2p-mdns

a79d57c

Remove default feature from libp2p-dns

148090f

Remove default feature from libp2p-tcp

8bf0ad7

Remove default feature from libp2p-uds

e6b49a3

Remove default-features = false from libp2p-wasm-ext

0a92268

This crate does not have any default features.

Remove default features from libp2p crate

c6f5090

Adds a CI check that ensures we specify the correct features for each example.

Put features onto one line

a31f595

Introduce full feature

37c7c79

Align how we specify the path to libp2p across workspace

c5968d6

Add CI check to enforce that full feature contains all features

cbb66a5

Use full feature for all examples

9add85b

This gets rid of the cool but complicated CI check.

thomaseizinger requested review from mxinden and elenaf9 September 20, 2022 03:54

thomaseizinger added 7 commits September 20, 2022 13:54

Fill in PR number

844adb5

test on bash needs quotes

4b8091d

Full in missing changelog entries for libp2p-metrics

26bf2b8

Enable full feature for integration tests

7f0a54c

Add missing link to PR

0bf360b

Add missing entries for autonat

fbee01e

Put links in correct order

32978fd

thomaseizinger mentioned this pull request Sep 21, 2022

transports/onion: Add dial-only implementation of Transport #2899

Closed

4 tasks

mxinden reviewed Sep 21, 2022

View reviewed changes

Update CHANGELOG.md

eaea056

Co-authored-by: Max Inden <mail@max-inden.de>

thomaseizinger added 2 commits September 22, 2022 19:13

Merge branch 'master' into 2173-no-default-features

58fa8ba

Merge branch '2173-no-default-features' of github.com:libp2p/rust-lib…

feb7c32

…p2p into 2173-no-default-features

Merge branch 'master' into 2173-no-default-features

4f7010a

thomaseizinger mentioned this pull request Sep 23, 2022

Re-design feature sets #2173

Closed

4 tasks

thomaseizinger added 4 commits September 28, 2022 10:28

Merge branch 'master' into 2173-no-default-features

dbaf661

Fix bad merge

fc94c06

Re-format changelog and remove duplicates

2450831

Fix bad link

8af0768

mxinden mentioned this pull request Sep 29, 2022

transports/quic: Add implementation based on quinn-proto #2289

Merged

7 tasks

Merge branch 'master' into 2173-no-default-features

fa6e027

mxinden merged commit f6bb846 into master Sep 29, 2022

thomaseizinger deleted the 2173-no-default-features branch September 29, 2022 22:34

thomaseizinger mentioned this pull request Sep 30, 2022

{core,muxers/mplex}: Fix bad version bump #2960

Merged

4 tasks

elenaf9 added a commit to elenaf9/rust-libp2p that referenced this pull request Oct 2, 2022

transport/quic: Remove default feature

1e26fec

See libp2p#2918.

elenaf9 added a commit to kpp/rust-libp2p that referenced this pull request Oct 3, 2022

transport/quic: Remove default feature

ebc2c9a

See libp2p#2918.

This was referenced Oct 4, 2022

Properly set rustdoc attr. Closing #2950 #2975

Closed

*: Updated comments in examples #2985

Merged

thomaseizinger mentioned this pull request Oct 14, 2022

fix: Remove circular dependencies across workspace #3023

Merged

4 tasks

thomaseizinger mentioned this pull request Oct 23, 2022

Avoid circular dependencies in workspace #3053

Closed

mxinden mentioned this pull request Nov 2, 2022

Fix swarm changelog entry #3063

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: Remove default features from all crates #2918

*: Remove default features from all crates #2918

thomaseizinger commented Sep 20, 2022

mxinden left a comment

mxinden Sep 21, 2022

thomaseizinger Sep 22, 2022 •

edited

mxinden Sep 22, 2022

thomaseizinger Oct 5, 2022

mxinden Oct 5, 2022

thomaseizinger Oct 14, 2022

thomaseizinger commented Sep 22, 2022

thomaseizinger commented Sep 23, 2022

dignifiedquire commented Sep 23, 2022

thomaseizinger commented Sep 23, 2022

dignifiedquire commented Sep 23, 2022

mxinden commented Sep 27, 2022

thomaseizinger commented Sep 28, 2022

thomaseizinger commented Sep 28, 2022

thomaseizinger commented Sep 29, 2022

umgefahren commented Oct 4, 2022

*: Remove default features from all crates #2918

*: Remove default features from all crates #2918

Conversation

thomaseizinger commented Sep 20, 2022

Description

Links to any relevant issues

Open Questions

Change checklist

mxinden left a comment

Choose a reason for hiding this comment

mxinden Sep 21, 2022

Choose a reason for hiding this comment

thomaseizinger Sep 22, 2022 • edited

Choose a reason for hiding this comment

Footnotes

mxinden Sep 22, 2022

Choose a reason for hiding this comment

thomaseizinger Oct 5, 2022

Choose a reason for hiding this comment

mxinden Oct 5, 2022

Choose a reason for hiding this comment

thomaseizinger Oct 14, 2022

Choose a reason for hiding this comment

thomaseizinger commented Sep 22, 2022

thomaseizinger commented Sep 23, 2022

dignifiedquire commented Sep 23, 2022

thomaseizinger commented Sep 23, 2022

dignifiedquire commented Sep 23, 2022

mxinden commented Sep 27, 2022

thomaseizinger commented Sep 28, 2022

thomaseizinger commented Sep 28, 2022

thomaseizinger commented Sep 29, 2022

umgefahren commented Oct 4, 2022

thomaseizinger Sep 22, 2022 •

edited