Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MaterializeIfSomeParentsUpdated AutoMaterializeRule #21600

Open
ion-elgreco opened this issue May 2, 2024 · 0 comments
Open

Add MaterializeIfSomeParentsUpdated AutoMaterializeRule #21600

ion-elgreco opened this issue May 2, 2024 · 0 comments
Labels

Comments

@ion-elgreco
Copy link
Contributor

What's the use case?

Usually I have assets which have multiple upstream asset inputs. I want to be able to materialize them only if at least some subset of the upstream assets collectively got updated. The requirement is not on all assets, just a subset.

As far as I can see it's not possible to do this yet.

Ideas of implementation

Add new rule, where you can specify the asset names.

If we need to specify the asset key, it's very important when doing load_assets_from_modules that also the asset keys in this rule will be prefixed with the key when provided.

We often have all assets without a prefix until runtime, when we prefix them with the current branch.

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

@ion-elgreco ion-elgreco changed the title Add SkipIfSomeParentsNotUpdated rule Add MaterializeIfSomeParentsUpdated rule May 2, 2024
@garethbrickman garethbrickman added the area: auto-materialize Related to Auto Materialization label May 2, 2024
@garethbrickman garethbrickman changed the title Add MaterializeIfSomeParentsUpdated rule Add MaterializeIfSomeParentsUpdated AutoMaterializeRule May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants