Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dequeue minimum guaranteed logs as a priority #13179

Open
wants to merge 4 commits into
base: feature/AUTO-9357
Choose a base branch
from

Conversation

ferglor
Copy link
Collaborator

@ferglor ferglor commented May 10, 2024

https://smartcontract-it.atlassian.net/browse/AUTO-10165

In this PR, we're modifying the log provider to dequeue the minium number of guaranteed logs across all block windows as a priority, before comitting to best effort execution.

As an example, if we have 2 upkeeps and 3 block windows, each with 10 logs to be dequeued in total (5 per upkeep), and assuming a limit of 1 log per upkeep, we would:

  • Dequeue 2 logs from block window 1 # minimum commitment for block window 1 met
  • Dequeue 2 logs from block window 2 # minimum commitment for block window 2 met
  • Dequeue 2 logs from block window 3 # minimum commitment for block window 3 met

At this point, minimum commitment for all three block windows has been fulfilled, we can now operate on a best effort basis, going back to block window 1:

  • Dequeue 2 logs from block window 1
  • Dequeue 2 logs from block window 1
  • Dequeue 2 logs from block window 1
  • Dequeue 2 logs from block window 1 # all 10 logs have been dequeued from block window 1
  • Dequeue 2 logs from block window 2
  • Dequeue 2 logs from block window 2
  • Dequeue 2 logs from block window 2
  • Dequeue 2 logs from block window 2 # all 10 logs have been dequeued from block window 2
  • Dequeue 2 logs from block window 3
  • Dequeue 2 logs from block window 3
  • Dequeue 2 logs from block window 3
  • Dequeue 2 logs from block window 3 # all 10 logs have been dequeued from block window 3

@ferglor ferglor requested a review from a team as a code owner May 10, 2024 18:33
Copy link
Contributor

I see you updated files related to core. Please run pnpm changeset in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@ferglor ferglor changed the title Dequeue minimum guaranteed logs across as block windows first Dequeue minimum guaranteed logs as a priority May 10, 2024
@cl-sonarqube-production
Copy link

Quality Gate passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

latestBlockWindow, _, isWindowComplete := getBlockWindow(latestBlock, blockRate)

var dequeuedLatestCompleteWindow bool
// if the current latest window is incomplete, check that we have dequeued the second most recent window
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's ensure we dequeue logs from an incomplete window before moving to best effort as o/w we'll introduce delay for latest blocks

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more I think about this, I'm not sure how we can do this without introducing some delay for some upkeeps? E.g if we choose to dequeue from non-complete windows, imagine this scenario, for a 2 block window:

  • First block contains logs for upkeeps 1 and 2
  • Second block contains logs for upkeeps 2 and 3

If we choose to dequeue on an incomplete block window, when only the first block exists, we'll dequeue potenitally the min number of upkeeps for upkeeps 1 and 2, and this window will be considered as having its minimum commitment met, which would starve upkeep 3?

If we choose to dequeue on complete windows, then we would switch over to best effort mode until the block window was complete, and once both blocks were available, we'd dequeue min commitment for this window, before resuming best effort.

if !isWindowComplete {
penultimateBlockWindow, _, _ := getBlockWindow(latestBlock-int64(blockRate), blockRate)

if dequeuedPenultimate, ok := p.dequeuedMinimum[penultimateBlockWindow]; ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q. is this state reset by reorgs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants