Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update libp2p to v0.48 #661

Merged
merged 21 commits into from Oct 7, 2022
Merged

update libp2p to v0.48 #661

merged 21 commits into from Oct 7, 2022

Conversation

leviathanbeak
Copy link
Contributor

@leviathanbeak leviathanbeak commented Sep 30, 2022

Closes #530 #644.

Also this introduces dependency on protoc for compiling Protobuf files, previously prost library bundled protoc within the lib, but they've recently moved away from that approach.

Changes
libp2p has made some breaking changes in recent updates, main change was how you handle behaviour events - this means that the Behaviour, in our case FuelBehaviour, would be stateless apart from holding other inner behaviours and all the stateful logic would be moved to FuelP2PService that holds the FuelBehaviour.

Usually this would just mean a lot of cut & paste (check the service.rs and behaviour.rs files), but in our case FuelBehaviour was generic over a NetworkCodec, and leaving it still generic would imply that our FuelBehaviourEvent enum would also need to be generic which made for some unwieldy code.

I decided to create a wrapper struct

pub struct BehaviourEventWrapper<Codec: NetworkCodec> {
    event: FuelBehaviourEvent,
    codec: PhantomData<Codec>,
}

from this wrapper we would easily get the inner event and handle it in FuelP2PService.

Also now FuelP2PService needed to become generic over NetworkCodec since it's now the one that handles encoding and decoding of the network messages.

@leviathanbeak leviathanbeak marked this pull request as ready for review October 1, 2022 09:53
.github/workflows/ci.yml Show resolved Hide resolved
fuel-p2p/src/service.rs Outdated Show resolved Hide resolved
fuel-p2p/src/service.rs Show resolved Hide resolved
fuel-p2p/src/orchestrator.rs Show resolved Hide resolved
.github/workflows/ci.yml Show resolved Hide resolved
Copy link
Contributor

@freesig freesig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from what I understand. I think the mdns might be the wrong feature though

libp2p = { version = "0.44", default-features = false, features = [
"dns-async-std", "gossipsub", "identify", "kad", "mdns", "mplex", "noise",
libp2p = { version = "0.48", default-features = false, features = [
"dns-async-std", "gossipsub", "identify", "kad", "mdns-async-io", "mplex", "noise",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be mdns-tokio?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily, we currently use dns-async, tcp-async and mdn-async so should be fine, we might want to move them to tokio but previously it wasn't as stable as the async ones.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this be running as part of the fuel-core binary though? Which does use tokio. The problem with using the wrong runtime is you end up pulling in the whole async-io runtime which really increases compile times.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah man, for some silly reason I didn't think about these implementations that they would include different runtimes, that is bad, not only on compile times but having 2 different async runtimes is inviting trouble - so yeah I'll move to tokio ones, thanks

fuel-p2p/src/orchestrator.rs Show resolved Hide resolved
fuel-p2p/src/discovery/discovery_config.rs Outdated Show resolved Hide resolved
@leviathanbeak leviathanbeak self-assigned this Oct 3, 2022
@leviathanbeak leviathanbeak added fuel-p2p enhancement New feature or request labels Oct 3, 2022
fuel-p2p/src/config.rs Outdated Show resolved Hide resolved
@Voxelot
Copy link
Member

Voxelot commented Oct 4, 2022

This issue existed before the upgrade, but when I startup fuel-core with p2p enabled:

cargo run -p fuel-core --features p2p -- run --network test --enable_mdns

and then use ctrl-c to exit, I get a bunch of these errors:

2022-10-04T03:01:20.376180Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376216Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376254Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376295Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376337Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376349Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376383Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376425Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376437Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376445Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376482Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376495Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376502Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376540Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376579Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376617Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent
2022-10-04T03:01:20.376655Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent

@leviathanbeak
Copy link
Contributor Author

This issue existed before the upgrade, but when I startup fuel-core with p2p enabled:

cargo run -p fuel-core --features p2p -- run --network test --enable_mdns

and then use ctrl-c to exit, I get a bunch of these errors:

2022-10-04T03:01:20.376180Z  WARN fuel-libp2p: 153: Failed to receive P2PRequestEvent

...
@Voxelot P2PRequestEvent is the Event that Network Orchestrator receives from other fuel-core modules.
Meaning, this is not related to p2p itself, but the request is coming from one of the other modules and it seems it's not handled in a best way possible. Network Orchestrator is receiving None as a request while shutting down - this would be more of a task for PRs similar to #590

Copy link
Contributor

@freesig freesig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand this code but with that in mind it looks good.

@leviathanbeak leviathanbeak dismissed ControlCplusControlV’s stale review October 7, 2022 10:09

dismissing since all has been marked as resolved by the reviewer

@leviathanbeak leviathanbeak merged commit 15e097e into master Oct 7, 2022
@leviathanbeak leviathanbeak deleted the leviathanbeak/update_libp2p branch October 7, 2022 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request fuel-p2p
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RUSTSEC-2022-0040: Multiple soundness issues in owning_ref
5 participants