Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use efficient data structure in StreamFrameSorter #300

Open
marten-seemann opened this issue Aug 22, 2016 · 8 comments · May be fixed by #3471
Open

use efficient data structure in StreamFrameSorter #300

marten-seemann opened this issue Aug 22, 2016 · 8 comments · May be fixed by #3471
Labels

Comments

@marten-seemann
Copy link
Member

This is the follow-up of #298.

I increased the MaxStreamFrameSorterGaps from 50 to 1000, which makes us (a bit) vulnerable to a DoS attack if a malicious client sends us StreamFrames with a lot of gaps. It should not be too critical, for comparison, Chrome recently introduced a limit of 10000 here. It's still something we should mitigate some time soon.

We need some data structure here that allows for binary searches as well as random inserts and removal of elements, so probably some kind of (balanced) tree structure.

@marten-seemann marten-seemann added this to the 0.5 milestone Aug 22, 2016
@marten-seemann marten-seemann modified the milestones: 0.6, 0.5 Nov 2, 2016
@marten-seemann
Copy link
Member Author

Since we'll soon accept overlapping stream data (#408), we might need to think again about the best data structure to use here. However, I think #408 superseeds this issue. Closing.

@marten-seemann
Copy link
Member Author

Reopening.

@marten-seemann
Copy link
Member Author

It would be nice if the same data structure could also be used in the sentPacketHandler for keeping track of sent packets.
A requirement for this use case is that the tree supports modification while iterating (we might need to remove M out of N sent packets, and we don't want want to incur a M*log(N) cost for that.

Unfortunately, the only (?) popular generic tree data structure around is Google's Btree, which 1. has a horrible iterator interface and 2. doesn't support modifications while iterating.

It would be nice to find a 3rd party package that's widely used (let's say at least 100 imports, as reported by godoc), and actively maintained. While there are a few repos around, none of them even makes it to 10 imports, so I don't feel very comfortable relying on their code.

@zllovesuki
Copy link
Contributor

Can we use a concurrent skip[list,set,map] for this?

@marten-seemann
Copy link
Member Author

@zllovesuki Do you have a specific one in mind?
Would we create our own implementation, or is there a 3rd party library we can build on?

@zllovesuki
Copy link
Contributor

zllovesuki commented Jan 26, 2023

@marten-seemann was thinking about https://github.com/zhangyunhao116/skip[set,map] implementation, purely because I did something similar to frameSorter in Java with ConcurrentSkipListSet.

As for import counts, let's just say that private repository imports don't show up

@zllovesuki
Copy link
Contributor

There exists a lock-free RBT paper but I don't think there's an implementation yet: http://www.cs.umanitoba.ca/%7Ehacamero/Research/RBTreesKim.pdf

@marten-seemann
Copy link
Member Author

This probably doesn't directly apply to the stream frame sorter, but here's a proposal for an efficient way to track sent packets and handle ACKs: #3836

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants