Discussion: Support planned pausing

[hannahhoward]

Goals

Right now, our pause mechanism in retrieval requires inspecting every block and telling graphsync whether you want to pause, and executing this synchronously. It is less than ideal to say the least. A better mechanism would be for the retrieval protocol to tell graphsync ahead of time when to pause. This becomes especially neccesary if we move to a mode where the payments might be happening in a different process -- and checking with them is an HTTP call (that we definitely would prefer not to make on each call)

Note that we could implement this entirely in go-data-transfer without these changes in Graphsync. see filecoin-project/go-data-transfer#297

That said, this seems like it'd make it easier.

For now, I see this working like as follows:

Add an action for the IncomingRequestHook:

type PlannedPause struct {
    PauseAfterBytesSent uint64
    PauseIfComplete bool
    OnPause(actualSent uint64) []graphsync.ExtensionData
}

type IncomingRequestHookActions interface {
    // maintain existing actions
    PlanPause(PlannedPause)
}

type UpdateRequestHookActions interface {
   // maintain existing actions
   PlanPause(PlannedPause)
}

Note that multiple hooks may create planned pauses so we may need to keep a sorted array of pauses.

Logic is after each block, check pause requests, if any are before the current pause point, call on OnPause for all that have passed, add the extensions to the response, and pause it.

[rvagg]

what does it mean to "pause if complete"? isn't that just a stop? or is that there just to provide an option to get that callback when complete?

[hannahhoward]

I mean it's considering the request "not done" -- this is one of the more bizarre aspects of the retrieval protocol, where the request is "stopped" pending the last payment, even though the other party has all their data and can easily walk away. Essentially it allows the retriever to send one last request for payment.

[willscott]

• in that OnPause callback how would you signal an 'actually don't pause'?
• is having one of these calculated at each future byte offset particularly efficient either?