Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(p2p/discovery)!: Implement archival discovery + syncing of historic blocks and blobs #3188

Merged
merged 28 commits into from
May 28, 2024

Conversation

renaynay
Copy link
Member

@renaynay renaynay commented Feb 15, 2024

This PR introduces the discovery of nodes advertising on the archival topic for the purpose of being able to request blobs and blocks that are older than the sampling window.

In order to do so, there are now 2 instances of discovery (one for full nodes -- all nodes running full block sync within the sampling window, and one for archival nodes -- nodes that sync, store and serve all historic data) and 2 instances of peer managers (one for full and one for archival).

This PR is breaking due to changes to metrics names.

Please also note that this PR introduces the ability to perform archival sync of blobs + blocks for LIGHT AND FULL nodes only regardless of whether the full nodes have pruning enabled or not. Bridge nodes that have previously had pruning enabled will NOT be able to request historic data from the consensus core node. The bridge node must run in archival mode (without --experimental-pruning enabled) in order to access historic data.

Bridge and Full nodes in archival mode (without --experimental-pruning)

2024-05-10T15:21:44.490+0200	INFO	share/discovery	discovery/manager.go:30	starting discovery	{"topic": "full"}
2024-05-10T15:21:44.491+0200	INFO	share/discovery	discovery/manager.go:33	advertising to topic	{"topic": "full"}
2024-05-10T15:21:44.491+0200	INFO	share/discovery	discovery/manager.go:30	starting discovery	{"topic": "archival"}
2024-05-10T15:21:44.491+0200	INFO	share/discovery	discovery/manager.go:33	advertising to topic	{"topic": "archival"}

Bridge and Full nodes running --experimental-pruning

2024-05-10T15:19:20.698+0200	INFO	share/discovery	discovery/manager.go:30	starting discovery	{"topic": "full"}
2024-05-10T15:19:20.698+0200	INFO	share/discovery	discovery/manager.go:33	advertising to topic	{"topic": "full"}
2024-05-10T15:19:20.698+0200	INFO	share/discovery	discovery/manager.go:30	starting discovery	{"topic": "archival"}

Light nodes

2024-05-10T15:20:42.440+0200	INFO	share/discovery	discovery/manager.go:30	starting discovery	{"topic": "full"}
2024-05-10T15:20:42.440+0200	INFO	share/discovery	discovery/manager.go:30	starting discovery	{"topic": "archival"}

TODO

  • test archival request routing per node type
  • renaming some things inside discovery away from being full-node-centric
  • rebase
  • swamp test

For follow-up PR:

  • clarify / make disc metrics generic as there will now be 2 instances of discovery happening

Nice to have;

  • benchmarks for LN

@renaynay renaynay added area:p2p area:storage kind:feat Attached to feature PRs labels Feb 15, 2024
@renaynay renaynay force-pushed the shrex-getter-routing branch 2 times, most recently from 50dfce3 to 6e201c8 Compare February 27, 2024 15:33
@renaynay renaynay self-assigned this Mar 7, 2024
@renaynay renaynay force-pushed the shrex-getter-routing branch 2 times, most recently from 15a2d9f to 7acc848 Compare March 18, 2024 09:28
@renaynay renaynay changed the title [WIP] Archival discovery + sync feat(p2p/discovery): Implement archival discovery + syncing of historic blocks and blobs Mar 28, 2024
renaynay added a commit that referenced this pull request Apr 18, 2024
Subscription was never cancelled on Stop, found while working on swamp
tests for #3188
Wondertan pushed a commit that referenced this pull request Apr 21, 2024
Subscription was never cancelled on Stop, found while working on swamp
tests for #3188
@renaynay renaynay force-pushed the shrex-getter-routing branch 2 times, most recently from 3764930 to d79a9c7 Compare May 6, 2024 18:52
@renaynay renaynay added the v0.14.0 Intended for v0.14.0 release label May 7, 2024
@renaynay renaynay force-pushed the shrex-getter-routing branch 2 times, most recently from 1684a62 to e276af8 Compare May 8, 2024 09:55
@renaynay renaynay changed the title feat(p2p/discovery): Implement archival discovery + syncing of historic blocks and blobs feat(p2p/discovery)!: Implement archival discovery + syncing of historic blocks and blobs May 8, 2024
@renaynay renaynay added the kind:break! Attached to breaking PRs label May 8, 2024
@renaynay renaynay marked this pull request as ready for review May 8, 2024 14:52
Copy link
Member

@walldiss walldiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't tell if nodebuilder is correct. Changes are quite complex and difficult to fully understand. So relying on tests for correctness. Not a good thing, so we should prioritise refactoring of share module construction.

das/daser.go Outdated Show resolved Hide resolved
nodebuilder/pruner/module.go Outdated Show resolved Hide resolved
nodebuilder/pruner/module.go Outdated Show resolved Hide resolved
nodebuilder/share/constructors.go Outdated Show resolved Hide resolved
nodebuilder/share/constructors.go Outdated Show resolved Hide resolved
share/availability/full/availability.go Show resolved Hide resolved
nodebuilder/tests/prune_test.go Outdated Show resolved Hide resolved
share/p2p/discovery/manager.go Outdated Show resolved Hide resolved
share/p2p/discovery/manager.go Outdated Show resolved Hide resolved
share/p2p/discovery/metrics.go Outdated Show resolved Hide resolved
Copy link
Member

@Wondertan Wondertan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two main comments. The introduced Manager is net negative and the need for EnableAdvertise in config is unclear.

share/p2p/discovery/options.go Outdated Show resolved Hide resolved
nodebuilder/share/module.go Outdated Show resolved Hide resolved
nodebuilder/share/module.go Outdated Show resolved Hide resolved
share/p2p/discovery/manager.go Outdated Show resolved Hide resolved
share/getters/shrex.go Outdated Show resolved Hide resolved
share/getters/options.go Outdated Show resolved Hide resolved
share/getters/shrex.go Outdated Show resolved Hide resolved
nodebuilder/share/module.go Show resolved Hide resolved
nodebuilder/share/module.go Show resolved Hide resolved
nodebuilder/share/module.go Show resolved Hide resolved
nodebuilder/share/opts.go Show resolved Hide resolved
share/getters/options.go Outdated Show resolved Hide resolved
share/p2p/discovery/manager.go Outdated Show resolved Hide resolved
share/p2p/peers/options.go Outdated Show resolved Hide resolved
…nt of both peermans to shrex and remove from nodebuilder, require availWindow as arg for shrex ctor
Wondertan
Wondertan previously approved these changes May 27, 2024
Copy link
Member

@Wondertan Wondertan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to ship pruning and unblock this PR, so approving,
Under different circumstances, I would request a redesign.
This PR leaves too much dech debt behind it

Copy link
Member

@walldiss walldiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great effort on the nodebuilder in this PR! However, it's still pretty tough to tell if everything's correct. My biggest concern is that the feature needs thorough testing to ensure all setups work as intended. Given its complexity, future changes might get tricky, so let's prioritize refactoring as soon as we have the resources.

@renaynay renaynay enabled auto-merge (squash) May 28, 2024 11:02
@renaynay renaynay merged commit 7848c22 into celestiaorg:main May 28, 2024
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:p2p area:storage kind:break! Attached to breaking PRs kind:feat Attached to feature PRs v0.14.0 Intended for v0.14.0 release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants