Go IDONTWANT #553

ppopth · 2024-01-15T18:32:36Z

Sending IDONTWANT

Implement a smart queue
Add priorities to the smart queue
Put IDONTWANT packets into the smart priority queue as soon as the node gets the packets

Handling IDONTWANT

Use a map to remember the message ids whose IDONTWANT packets have been received
Implement max_idontwant_messages (ignore the IDONWANT packets if the max is reached)
Clear the message IDs from the cache after 3 heartbeats
Hash the message IDs before putting them into the cache.

Since we want to implement a priority queue later, we need to replace the normal sending channels with the new smart structures first.

vyzo

So far looking good overall.

However, I think using a slice to back the queue is problematic at many levels as it might leak space with a growing array, and also hang on to too much space.

I would very much prefer using a dequeue for this.

vyzo · 2024-02-10T10:28:12Z

comm.go

-			}
+		rpc, err := outgoing.Pop(ctx)
+		if err != nil {
+			return


We might want to log the error (at very low priority) if it is not a closed indicator.

rpc_queue.go

UrgentPush allows you to push an rpc packet to the front of the queue so that it will be popped out fast.

Most importantly, this commit adds a new method called PreValidation to the interface PubSubRouter, which will be called right before validating the gossipsub message. In GossipSubRouter, PreValidation will send the IDONTWANT controll messages to all the mesh peers of the topics of the received messages.

ppopth · 2024-03-02T06:17:03Z

However, I think using a slice to back the queue is problematic at many levels as it might leak space with a growing array, and also hang on to too much space.

The unused space will be freed eventually https://stackoverflow.com/a/26863706

This reverts commit dfd81e8.

ppopth · 2024-03-05T09:47:36Z

I'm not really sure if we need dfd81e8 or not. I guess adding a non-standard flag to pb/trace.proto would break some users of the tracers?

vyzo · 2024-03-22T08:58:36Z

I'm not really sure if we need dfd81e8 or not. I guess adding a non-standard flag to pb/trace.proto would break some users of the tracers?

Yeah, let's not break compatibility with existing code, we don't need it.

Any progress other than that?

ppopth · 2024-04-08T09:51:36Z

Any progress other than that?

I haven't implemented the second half. Does the first half looks good to you?

vyzo · 2024-04-08T09:58:08Z

yes, looks reasonable so far.

When receiving IDONTWANTs, the host should remember the message ids contained in IDONTWANTs using a hash map. When receiving messages with those ids, it shouldn't forward them to the peers who already sent the IDONTWANTs. When the maximum number of IDONTWANTs is reached for any particular peer, the host should ignore any excessive IDONTWANTs from that peer.

ppopth · 2024-05-01T18:07:49Z

@vyzo I finished everything already. Sorry for the delay. I was in vacation and was a bit busy lately. Thank you for your patience.

vyzo

Generally looks good, but lets make the counter clear less aggressive.

vyzo · 2024-05-02T10:52:48Z

gossipsub.go

@@ -56,7 +56,9 @@ var (
 	GossipSubGraftFloodThreshold              = 10 * time.Second
 	GossipSubMaxIHaveLength                   = 5000
 	GossipSubMaxIHaveMessages                 = 10
+	GossipSubMaxIDontWantMessages             = 10


isnt that too low?

Yeah, I didn't think enough about this. I just copied it from GossipSubMaxIHaveMessages. I think I will change it to 1,000 which will makes the IDONTWANT cache grows by 60,000 per minute (because GossipSubHeartbeatInterval is 1 second)

vyzo · 2024-05-02T10:56:16Z

gossipsub.go

@@ -1617,6 +1693,13 @@ func (gs *GossipSubRouter) clearIHaveCounters() {
 	}
 }

+func (gs *GossipSubRouter) clearIDontWantCounters() {


Thats too aggressive and can clear the counters at unfortunate times.
Lets keep them for heartbeat or two, maybe 3 to match the cache sliding window.

I don't understand this. When is the unfortunate time?

This only clear the counters not the cache that keeps the message ids from IDONTWANTs. So technically, the slower you clear the cache, the more you prevent the DoS attack. The faster you clear the cache, the more efficiency you have (because you process more IDONTWANTs).

Lets say you just got an IDW, heartbeat clear, and then the message arrives... and you send it even though it is not wanted.

and you send it even though it is not wanted.

This is not true.

The heartbeat clear only clears the counter, not the cache. The IDONTWANT is still in the cache so the message is not forwarded. In other words, the heartbeat clear clears peerdontwant, while only unwanted is used to decide whether to forward it or not.

when is the cache cleared?

you misunderatand.

We need to keep the IDW for some heartbeats and then clear them.

It is both for controlling memory use and for correctness.

reagardless, an LRA (least recently added) cache for IDW could work and keep memory bounded and it is possibly simpler.

However we need to consider cache sizes carefully.

and the problem here is a sybil attack, and we get to keep all the garbage indefinitely.

So no, this is not the correct solution.

We need to explicitly clean old IDWs.

In fact we should go one step further and use a global limit for IDWs across all peers in the cache.

It is both for controlling memory use and for correctness.

Alright, got it. Thanks.

ppopth · 2024-05-03T17:25:28Z

The message ids from IDONTWANT can be very large, so hashing it before keeping it in memory quite makes sense. The same thing is already implemented in nim-libp2p vacp2p/nim-libp2p#1090

vyzo · 2024-05-03T17:28:58Z

sure, but it needs to be cleared, otherwise its memroy leak and dos waiting to happen. Or at best the feature breaks itself for long running nodes.

I suggest for each IDW keep a counter of how many heartbeats it survived, and after 3 (to match the IHAVE ads) it should be cleared.

Replace sending channel with the smart rpcQueue

3b4d97b

Since we want to implement a priority queue later, we need to replace the normal sending channels with the new smart structures first.

vyzo self-requested a review January 15, 2024 18:34

vyzo reviewed Feb 10, 2024

View reviewed changes

ppopth added 6 commits February 22, 2024 08:47

Implement UrgentPush in the smart rpcQueue

773c26f

UrgentPush allows you to push an rpc packet to the front of the queue so that it will be popped out fast.

Add IDONTWANT to rpc.proto and trace.proto

1f36b9d

Add urgent flag to SendRPC in trace.proto

dfd81e8

Test GossipSub IDONWANT sending

055ffa0

Send IDONWANT only for large messages

6b24d35

Revert "Add urgent flag to SendRPC in trace.proto"

ec6ded9

This reverts commit dfd81e8.

vyzo reviewed May 2, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Go IDONTWANT #553

Go IDONTWANT #553

ppopth commented Jan 15, 2024 •

edited

vyzo left a comment

vyzo Feb 10, 2024

ppopth commented Mar 2, 2024

ppopth commented Mar 5, 2024

vyzo commented Mar 22, 2024

ppopth commented Apr 8, 2024

vyzo commented Apr 8, 2024

ppopth commented May 1, 2024

vyzo left a comment

vyzo May 2, 2024

ppopth May 2, 2024 •

edited

vyzo May 2, 2024

ppopth May 2, 2024

vyzo May 2, 2024 •

edited

ppopth May 3, 2024

vyzo May 3, 2024

vyzo May 4, 2024

vyzo May 4, 2024 •

edited

vyzo May 4, 2024

vyzo May 4, 2024

ppopth May 4, 2024

ppopth commented May 3, 2024

vyzo commented May 3, 2024 •

edited

Go IDONTWANT #553

Are you sure you want to change the base?

Go IDONTWANT #553

Conversation

ppopth commented Jan 15, 2024 • edited

vyzo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ppopth commented Mar 2, 2024

ppopth commented Mar 5, 2024

vyzo commented Mar 22, 2024

ppopth commented Apr 8, 2024

vyzo commented Apr 8, 2024

ppopth commented May 1, 2024

vyzo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ppopth May 2, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vyzo May 2, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vyzo May 4, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ppopth commented May 3, 2024

vyzo commented May 3, 2024 • edited

ppopth commented Jan 15, 2024 •

edited

ppopth May 2, 2024 •

edited

vyzo May 2, 2024 •

edited

vyzo May 4, 2024 •

edited

vyzo commented May 3, 2024 •

edited