You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should push Artifacts to cover as many cases as possible, but explicit "side effect" Producers might be a good pressure release for interacting with external systems, for example:
Slack notifications (though a MessageArtifact is conceivable)
triggering external jobs/publish (though we should formalize Artifact publishing + hooks)
These producers might come in a few flavors:
"Run always" - run anytime a Graph is built, as long as the input Artifact(s) are built
"Run once per input(s)" - run anytime the input Artifacts are changed
A simple "syntactic sugar" implementation might just add an implicit "checkpoint" output Artifact to the Producer. "Run always" might have a random number/timestamp generator for Producer.version so the output is always "stale". "Run once per input(s)" might want to choose between "at least once" and "at most once" delivery (Artifacts writes are generally "at least once", but "at most once" might be more important for external notifications, though requiring more coordination).
Normal Producer output gets assigned to the Graph (which is how they're identified), but won't be natural for these side effect Producers (no return). Perhaps there's an additional method to register them: graph.delay(SendSlack(graph.artifacts.quality_report))? I'd rather avoid "global magic in with Graph(...) and implicit Producers self-registration" (1. Producers then have to know about Graph, 2. Producer can be used outside a Graph, meaning we can't know/warn accidentally initialize after the with Graph context). These Producers should be used more sparingly, so it's ok if they require a bit more to hook up.
The text was updated successfully, but these errors were encountered:
We should push
Artifacts
to cover as many cases as possible, but explicit "side effect" Producers might be a good pressure release for interacting with external systems, for example:These producers might come in a few flavors:
A simple "syntactic sugar" implementation might just add an implicit "checkpoint" output Artifact to the Producer. "Run always" might have a random number/timestamp generator for
Producer.version
so the output is always "stale". "Run once per input(s)" might want to choose between "at least once" and "at most once" delivery (Artifacts writes are generally "at least once", but "at most once" might be more important for external notifications, though requiring more coordination).Normal
Producer
output gets assigned to the Graph (which is how they're identified), but won't be natural for these side effectProducers
(no return). Perhaps there's an additional method to register them:graph.delay(SendSlack(graph.artifacts.quality_report))
? I'd rather avoid "global magic inwith Graph(...)
and implicit Producers self-registration" (1. Producers then have to know about Graph, 2. Producer can be used outside a Graph, meaning we can't know/warn accidentally initialize after thewith Graph
context). These Producers should be used more sparingly, so it's ok if they require a bit more to hook up.The text was updated successfully, but these errors were encountered: