Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockservice & exchange & bitswap: add non variadic NotifyNewBlock #242

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Jorropo
Copy link
Contributor

@Jorropo Jorropo commented Mar 30, 2023

Variadicts in go are just syntactic sugar around passing a slice, that means all go memory reachability rules apply, this force the compiler to heap allocate the variadic slice for virtual call, because the implementation is allowed to leak the slice (and go's interprocedural optimisations do not cover virtuals).

Passing a block without variadic will pass the itab either on the stack or decomposed through registers. Skipping having to allocate a slice.

@Jorropo Jorropo self-assigned this Mar 30, 2023
@Jorropo Jorropo requested a review from a team as a code owner March 30, 2023 16:48
@Jorropo Jorropo force-pushed the notify-new-block branch 3 times, most recently from 5344f5d to 4ff4bd2 Compare March 30, 2023 17:02
@codecov
Copy link

codecov bot commented Mar 30, 2023

Codecov Report

Merging #242 (311cd1f) into main (085ed9d) will decrease coverage by 0.09%.
The diff coverage is 91.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #242      +/-   ##
==========================================
- Coverage   48.23%   48.15%   -0.09%     
==========================================
  Files         270      270              
  Lines       33064    33080      +16     
==========================================
- Hits        15950    15930      -20     
- Misses      15455    15481      +26     
- Partials     1659     1669      +10     
Impacted Files Coverage Δ
blockservice/blockservice.go 73.15% <50.00%> (-0.42%) ⬇️
bitswap/bitswap.go 40.65% <100.00%> (+3.45%) ⬆️
bitswap/client/client.go 82.55% <100.00%> (-2.03%) ⬇️
bitswap/server/server.go 52.31% <100.00%> (+0.11%) ⬆️
examples/gateway/proxy/blockstore.go 50.66% <100.00%> (+1.35%) ⬆️
exchange/offline/offline.go 88.88% <100.00%> (+1.01%) ⬆️

... and 9 files with indirect coverage changes

Comment on lines +16 to +17
// NotifyNewBlock tells the exchange that a new block is available and can be served.
NotifyNewBlock(ctx context.Context, blocks blocks.Block) error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea how widespread the dependency on this interface (and therefore the result of the breakage) is and the benefits of implementing this? For example, does this show up at all in profiles as a meaningful thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think everyone relies either on the offline exchange (in order to wrap a blockstore) or bitswap. I don't belive many implementations exists, even in outside codebases.

Even if someone else implement this, just forwarding the call to NotifyNewBlocks with a slice of 1, this is legal and easy to write.
This just aims to get over golang's lacking interprocedural virtual optimisations, it does not change any feature.

Most of our other APIs have both single and multiple versions for this reason (Put vs PutMany, GetBlock vs GetBlocks, ...).

@BigLep
Copy link
Contributor

BigLep commented Mar 30, 2023

@Jorropo : I don't see an issue linked to this PR. In the absence of this, can you share more on what's motivating this PR?

@Jorropo
Copy link
Contributor Author

Jorropo commented Mar 30, 2023

@BigLep the PR and Commit bodies:

Variadicts in go are just syntactic sugar around passing a slice, that means all go memory reachability rules apply, this force the compiler to heap allocate the variadic slice for virtual call, because the implementation is allowed to leak the slice (and go's interprocedural optimisations do not cover virtuals).
Passing a block without variadic will pass the itab either on the stack or decomposed through registers. Skipping having to allocate a slice.

I can add the explanation about keeping it in-line with our other exchange APIs.


This seems like a good usecase to try the idea @guseggert is cooking (some automated code migrator), that would add the NotifyNewBlock method (which redirects to NotifyNewBlocks) automatically if you run it.

Copy link
Member

@hacdias hacdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have strong opinions on this. It seems a relatively small change that can bring memory benefits - although arguably low. If others agree, I'm fine to have it merged.

FYSA: I rebased the PR and added the updated exchange for the gateway examples.

Variadicts in go are just syntactic sugar around passing a slice, that
means all go memory reachability rules apply, this force the compiler to
heap allocate the variadic slice for virtual call, because the
implementation is allowed to leak the slice (and go's interprocedural
optimisations do not cover virtuals).

Passing a block without variadic will pass the itab either on the stack
or decomposed through registers. Skipping having to allocate a slice.
@BigLep
Copy link
Contributor

BigLep commented Apr 6, 2023

2023-04-06 conversation: will defer for now. Want to have clarity on how we handle breaking changes in Boxo.

@Jorropo Jorropo removed their assignment Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: 🔎 In Review
Development

Successfully merging this pull request may close these issues.

None yet

5 participants