Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update libp2p from 0.50.0 to 0.53.1 #1379

Merged
merged 68 commits into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
91fb125
WIP
MitchTurner Sep 22, 2023
f7004e5
WIP
MitchTurner Sep 25, 2023
707eee5
Merge remote-tracking branch 'origin/master' into update-libp2p
MitchTurner Oct 2, 2023
98d92e6
Merge branch 'master' into update-libp2p
bvrooman Oct 6, 2023
ae3f78f
Update Cargo.lock
bvrooman Oct 6, 2023
16c9e42
Merge branch 'master' into update-libp2p
Dentosal Oct 9, 2023
e77f2ff
WIP
MitchTurner Oct 9, 2023
5e30e1f
Some fixes, but now we need to add several more behaviours to replace…
xgreenx Oct 10, 2023
0e1e297
Remove addresses from PeerInfo
MitchTurner Oct 17, 2023
2eb3060
Merge remote-tracking branch 'origin/master' into update-libp2p
MitchTurner Oct 17, 2023
9d5ac7e
Fix lock file
MitchTurner Oct 17, 2023
ac7808b
Merge remote-tracking branch 'origin/master' into update-libp2p
MitchTurner Nov 22, 2023
9ba837e
Fix removed read/write methods
MitchTurner Nov 24, 2023
7e8fd6a
Remove FastMessageId, among other cleanup
MitchTurner Nov 24, 2023
9c74512
Remove `TokioDnsConfig`, other cleanup
MitchTurner Nov 24, 2023
99e8d51
Remove a bunch of unused stuff, replace private function, etc
MitchTurner Nov 24, 2023
74f7159
WIP
MitchTurner Nov 24, 2023
c333c37
Add trait impls, bump version, other update stuff
MitchTurner Nov 24, 2023
4fdd8cd
Fix typing around poll code
MitchTurner Nov 25, 2023
7e26141
Get compiling with a bunch of TODOs
MitchTurner Nov 25, 2023
2c4ca7c
Use BytesCodec for reading raw bytes from socket
MitchTurner Nov 27, 2023
f61d5a1
Migrate from `Upgrade` traits to `ConnectionUpgrade` traits
MitchTurner Nov 27, 2023
741d353
Migrate more missed trait impls
MitchTurner Nov 27, 2023
8efb0c6
Replace missing handle call in discovery
MitchTurner Nov 27, 2023
82d86d7
Add missing trait methods
MitchTurner Nov 27, 2023
abb1850
Add trait methods to peer_report
MitchTurner Nov 27, 2023
c31f581
WIP: fix test errors
MitchTurner Nov 27, 2023
93357b9
Get tests compiling
MitchTurner Nov 28, 2023
ba4b3a3
Add extra instumentation
MitchTurner Nov 29, 2023
1ba608e
Fixed something
xgreenx Nov 29, 2023
ef58a70
Move behavior to avoid bug
MitchTurner Nov 30, 2023
9852266
WIP: Update `SwarmBuilder`
MitchTurner Dec 1, 2023
4fc93cc
Fix the peer_id in the service to use same as Swarm
MitchTurner Dec 1, 2023
8cdda2c
WIP: Fix discovery test
MitchTurner Dec 2, 2023
06d6be0
Push something
xgreenx Dec 4, 2023
d713ed6
Track connection closures
MitchTurner Dec 5, 2023
560a4ee
WIP: use our transport
MitchTurner Dec 7, 2023
1dcee3d
Use our transport with `SwarmBuilder`
MitchTurner Dec 7, 2023
04ebd3b
WIP modify peer_report handlers
MitchTurner Dec 8, 2023
2686ff9
Replace stupid either with select
MitchTurner Dec 8, 2023
0ac9225
Fix gossipsub tests by reordering sub-behaviors
MitchTurner Dec 11, 2023
8740bde
Ignore failing mdns test
MitchTurner Dec 12, 2023
ff3ebd6
Remove extra async stuff
MitchTurner Dec 12, 2023
cc137d9
WIP cleanup warnings
MitchTurner Dec 13, 2023
bfaf2bd
Fix other compilation warnings
MitchTurner Dec 13, 2023
c093509
Merge remote-tracking branch 'origin/master' into update-libp2p
MitchTurner Dec 13, 2023
06527c3
WIP: Move Heartbeat and Identify to FuelBehavior
MitchTurner Dec 14, 2023
e7683ed
Finish removing heartbeat and identify from peer report
MitchTurner Dec 14, 2023
d039644
Replace address truncation logic, fix warnings
MitchTurner Dec 14, 2023
5f66b04
Fixed checksum verification. Instead of having a manual upgrade for t…
xgreenx Dec 15, 2023
bb0aa0b
Fixed the codec deserialization
xgreenx Dec 15, 2023
87d1e6b
Sort deps
MitchTurner Dec 15, 2023
3b2e92e
Extend timeout because it would not run locally
MitchTurner Dec 15, 2023
a883d81
Refactor behavior event handler
MitchTurner Dec 15, 2023
e23317b
Fixed mdns behaviour
xgreenx Dec 15, 2023
6fb2a47
reorder helpers
MitchTurner Dec 15, 2023
128ec77
Merge branch 'master' into update-libp2p
MitchTurner Dec 15, 2023
48687d9
Use connection timeout from config
MitchTurner Dec 16, 2023
f5a0913
Cleanup
MitchTurner Dec 16, 2023
7b8b5fb
Remove unused dep
MitchTurner Dec 18, 2023
e54c02e
Remove unused dep, remove unused fields
MitchTurner Dec 18, 2023
a4fe8ac
Cleanup, add details to todo
MitchTurner Dec 18, 2023
1897d92
Cleanup
MitchTurner Dec 18, 2023
2eaa139
Update crates/fuel-core/src/p2p_test_helpers.rs
MitchTurner Dec 19, 2023
1dbf6c8
Fix typo, add issue to TODO
MitchTurner Dec 19, 2023
d68c2a0
Merge remote-tracking branch 'origin/master' into update-libp2p
MitchTurner Dec 20, 2023
44de252
Merge branch 'master' into update-libp2p
MitchTurner Dec 20, 2023
e946440
Remove `()`
MitchTurner Dec 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1,977 changes: 584 additions & 1,393 deletions Cargo.lock

Large diffs are not rendered by default.

31 changes: 15 additions & 16 deletions crates/services/p2p/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,36 +22,35 @@ fuel-core-types = { workspace = true, features = [
] }
futures = { workspace = true }
ip_network = "0.4"
libp2p = { version = "=0.50.0", default-features = false, features = [
libp2p = { version = "=0.52.3", default-features = false, features = [
"dns",
"gossipsub",
"identify",
"kad",
"macros",
"mdns",
"mplex",
"noise",
"noise",
"request-response",
"secp256k1",
"tcp",
"tokio",
"yamux",
"websocket",
] }
libp2p-core = "=0.38.0"
libp2p-dns = "=0.38.0"
libp2p-gossipsub = "=0.43.0"
libp2p-identify = "=0.41.0"
libp2p-kad = "=0.42.0"
libp2p-mdns = "=0.42.0"
libp2p-mplex = "=0.38.0"
libp2p-noise = "=0.41.0"
libp2p-core = "=0.40.1"
libp2p-dns = "=0.40.0"
libp2p-gossipsub = "=0.45.1"
libp2p-identify = "=0.43.0"
libp2p-kad = "=0.44.5"
libp2p-mdns = "=0.44.0"
libp2p-mplex = "=0.40.0"
libp2p-noise = "=0.43.1"
libp2p-prom-client = { workspace = true }
libp2p-request-response = "=0.23.0"
libp2p-swarm = "=0.41.1"
libp2p-tcp = "=0.38.0"
libp2p-websocket = "=0.40.0"
libp2p-yamux = "=0.42.0"
libp2p-request-response = "=0.25.1"
libp2p-swarm = "=0.43.4"
libp2p-tcp = "=0.40.0"
libp2p-websocket = "=0.42.1"
libp2p-yamux = "=0.44.1"
postcard = { workspace = true, features = ["use-std"] }
prometheus-client = { workspace = true }
rand = { workspace = true }
Expand Down
14 changes: 7 additions & 7 deletions crates/services/p2p/src/behavior.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,18 @@ use crate::{
use fuel_core_types::fuel_types::BlockHeight;
use libp2p::{
gossipsub::{
error::PublishError,
Gossipsub,
GossipsubEvent,
Behaviour as Gossipsub,
Event as GossipsubEvent,
MessageAcceptance,
MessageId,
PublishError,
},
request_response::{
Behaviour as RequestResponse,
Config as RequestResponseConfig,
Event as RequestResponseEvent,
ProtocolSupport,
RequestId,
RequestResponse,
RequestResponseConfig,
RequestResponseEvent,
ResponseChannel,
},
swarm::NetworkBehaviour,
Expand All @@ -51,7 +51,7 @@ pub enum FuelBehaviourEvent {

/// Handles all p2p protocols needed for Fuel.
#[derive(NetworkBehaviour)]
#[behaviour(out_event = "FuelBehaviourEvent")]
#[behaviour(to_swarm = "FuelBehaviourEvent")]
MitchTurner marked this conversation as resolved.
Show resolved Hide resolved
pub struct FuelBehaviour<Codec: NetworkCodec> {
/// Node discovery
discovery: DiscoveryBehaviour,
Expand Down
2 changes: 1 addition & 1 deletion crates/services/p2p/src/codecs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ use crate::{
ResponseMessage,
},
};
use libp2p::request_response::RequestResponseCodec;
use libp2p::request_response::Codec as RequestResponseCodec;
use std::io;

/// Implement this in order to handle serialization & deserialization of Gossipsub messages
Expand Down
21 changes: 9 additions & 12 deletions crates/services/p2p/src/codecs/postcard.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,11 @@ use futures::{
AsyncWriteExt,
};
use libp2p::{
core::{
upgrade::{
read_length_prefixed,
write_length_prefixed,
},
ProtocolName,
core::upgrade::{
read_length_prefixed,
write_length_prefixed,
},
request_response::RequestResponseCodec,
request_response::Codec as RequestResponseCodec,
};
use serde::{
Deserialize,
Expand Down Expand Up @@ -266,11 +263,11 @@ impl NetworkCodec for PostcardCodec {
#[derive(Debug, Clone)]
pub struct MessageExchangePostcardProtocol;

impl ProtocolName for MessageExchangePostcardProtocol {
fn protocol_name(&self) -> &[u8] {
REQUEST_RESPONSE_PROTOCOL_ID
}
}
// impl ProtocolName for MessageExchangePostcardProtocol {
// fn protocol_name(&self) -> &[u8] {
// REQUEST_RESPONSE_PROTOCOL_ID
// }
// }

#[cfg(test)]
mod tests {
Expand Down
20 changes: 12 additions & 8 deletions crates/services/p2p/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,26 @@ use libp2p::{
muxing::StreamMuxerBox,
transport::Boxed,
},
gossipsub::GossipsubConfig,
gossipsub::Config as GossipsubConfig,
identity,
identity::{
secp256k1::SecretKey,
Keypair,
},
mplex,
noise::{self,},
noise::{
Config as NoiseConfig,
{self,},
},
tcp::{
tokio::Transport as TokioTcpTransport,
Config as TcpConfig,
},
yamux,
Multiaddr,
PeerId,
Transport,
};
use libp2p_mplex::MplexConfig;
use libp2p_yamux::Config as YamuxConfig;
use std::{
collections::HashSet,
net::{
Expand Down Expand Up @@ -272,17 +276,17 @@ pub(crate) fn build_transport(
.upgrade(libp2p::core::upgrade::Version::V1);

let noise_authenticated = {
let dh_keys = noise::Keypair::<noise::X25519Spec>::new()
let dh_keys = identity::Keypair::new()
.into_authentic(&p2p_config.keypair)
.expect("Noise key generation failed");

noise::NoiseConfig::xx(dh_keys).into_authenticated()
NoiseConfig::xx(dh_keys).into_authenticated()
};

let multiplex_config = {
let mplex_config = mplex::MplexConfig::default();
let mplex_config = MplexConfig::default();

let mut yamux_config = yamux::YamuxConfig::default();
let mut yamux_config = YamuxConfig::default();
yamux_config.set_max_buffer_size(MAX_RESPONSE_SIZE);
libp2p::core::upgrade::SelectUpgrade::new(yamux_config, mplex_config)
};
Expand Down
42 changes: 12 additions & 30 deletions crates/services/p2p/src/config/fuel_authenticated.rs
MitchTurner marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use fuel_core_types::secrecy::Zeroize;
use futures::{
future,
AsyncRead,
Expand All @@ -9,10 +8,9 @@ use futures::{
use libp2p::{
core::UpgradeInfo,
noise::{
NoiseAuthenticated,
NoiseError,
NoiseOutput,
Protocol,
Config as NoiseConfig,
Error as NoiseError,
Output as NoiseOutput,
},
InboundUpgrade,
OutboundUpgrade,
Expand All @@ -26,43 +24,32 @@ pub(crate) trait Approver {
}

#[derive(Clone)]
pub(crate) struct FuelAuthenticated<A: Approver, P, C: Zeroize, R> {
noise_authenticated: NoiseAuthenticated<P, C, R>,
pub(crate) struct FuelAuthenticated<A: Approver> {
noise_authenticated: NoiseConfig,
approver: A,
}

impl<A: Approver, P, C: Zeroize, R> FuelAuthenticated<A, P, C, R> {
pub(crate) fn new(
noise_authenticated: NoiseAuthenticated<P, C, R>,
approver: A,
) -> Self {
impl<A: Approver> FuelAuthenticated<A> {
pub(crate) fn new(noise_authenticated: NoiseConfig, approver: A) -> Self {
Self {
noise_authenticated,
approver,
}
}
}

impl<A: Approver, P, C: Zeroize, R> UpgradeInfo for FuelAuthenticated<A, P, C, R>
where
NoiseAuthenticated<P, C, R>: UpgradeInfo,
{
type Info = <NoiseAuthenticated<P, C, R> as UpgradeInfo>::Info;
type InfoIter = <NoiseAuthenticated<P, C, R> as UpgradeInfo>::InfoIter;
impl<A: Approver> UpgradeInfo for FuelAuthenticated<A> {
type Info = <NoiseConfig as UpgradeInfo>::Info;
type InfoIter = <NoiseConfig as UpgradeInfo>::InfoIter;

fn protocol_info(&self) -> Self::InfoIter {
self.noise_authenticated.protocol_info()
}
}

impl<A, T, P, C, R> InboundUpgrade<T> for FuelAuthenticated<A, P, C, R>
impl<A, T> InboundUpgrade<T> for FuelAuthenticated<A>
where
NoiseAuthenticated<P, C, R>: UpgradeInfo
+ InboundUpgrade<T, Output = (PeerId, NoiseOutput<T>), Error = NoiseError>
+ 'static,
<NoiseAuthenticated<P, C, R> as InboundUpgrade<T>>::Future: Send,
T: AsyncRead + AsyncWrite + Send + 'static,
C: Protocol<C> + AsRef<[u8]> + Zeroize + Send + 'static,
A: Approver + Send + 'static,
{
type Output = (PeerId, NoiseOutput<T>);
Expand All @@ -84,14 +71,9 @@ where
}
}

impl<A, T, P, C, R> OutboundUpgrade<T> for FuelAuthenticated<A, P, C, R>
impl<A, T> OutboundUpgrade<T> for FuelAuthenticated<A>
where
NoiseAuthenticated<P, C, R>: UpgradeInfo
+ OutboundUpgrade<T, Output = (PeerId, NoiseOutput<T>), Error = NoiseError>
+ 'static,
<NoiseAuthenticated<P, C, R> as OutboundUpgrade<T>>::Future: Send,
T: AsyncRead + AsyncWrite + Send + 'static,
C: Protocol<C> + AsRef<[u8]> + Zeroize + Send + 'static,
A: Approver + Send + 'static,
{
type Output = (PeerId, NoiseOutput<T>);
Expand Down