Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate "side effect only" Producers #11

Open
JacobHayes opened this issue Mar 7, 2021 · 1 comment
Open

Investigate "side effect only" Producers #11

JacobHayes opened this issue Mar 7, 2021 · 1 comment
Labels
design Design and use cases required enhancement New feature or request

Comments

@JacobHayes
Copy link
Member

We should push Artifacts to cover as many cases as possible, but explicit "side effect" Producers might be a good pressure release for interacting with external systems, for example:

  • Slack notifications (though a MessageArtifact is conceivable)
  • triggering external jobs/publish (though we should formalize Artifact publishing + hooks)

These producers might come in a few flavors:

  • "Run always" - run anytime a Graph is built, as long as the input Artifact(s) are built
  • "Run once per input(s)" - run anytime the input Artifacts are changed

A simple "syntactic sugar" implementation might just add an implicit "checkpoint" output Artifact to the Producer. "Run always" might have a random number/timestamp generator for Producer.version so the output is always "stale". "Run once per input(s)" might want to choose between "at least once" and "at most once" delivery (Artifacts writes are generally "at least once", but "at most once" might be more important for external notifications, though requiring more coordination).

Normal Producer output gets assigned to the Graph (which is how they're identified), but won't be natural for these side effect Producers (no return). Perhaps there's an additional method to register them: graph.delay(SendSlack(graph.artifacts.quality_report))? I'd rather avoid "global magic in with Graph(...) and implicit Producers self-registration" (1. Producers then have to know about Graph, 2. Producer can be used outside a Graph, meaning we can't know/warn accidentally initialize after the with Graph context). These Producers should be used more sparingly, so it's ok if they require a bit more to hook up.

@JacobHayes JacobHayes added enhancement New feature or request design Design and use cases required labels Mar 7, 2021
@JacobHayes
Copy link
Member Author

Would Hooks cover many of these use cases?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design Design and use cases required enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant