Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional option for TimeWindowPartitionMapping between different granularity PartitionsDefinitions #21617

Open
mmutso-boku opened this issue May 3, 2024 · 0 comments
Labels

Comments

@mmutso-boku
Copy link

What's the use case?

While this #21380 is most welcome addition, I think there is still an use case, which is missing but would be very useful.

The specific use case I have in mind is as follows:
Asset A (Hourly) -> Asset B (Daily). The default mapping behaviour for this is that for Asset B 2024-05-01 partition, partitions [2024-05-01-00:00...2024-05-01-23:00] in Asset A must be materialized.

I would like to be able to shift it by an hour so that for Asset B's partition, in this example 2024-05-01, partitions [2024-05-01-01:00...2024-05-02-00:00] in Asset A must be materialized.

So when currently the offsets in TimeWindowPartitionMapping are in the units of the downstream asset, it would be very good to have an option to have the offsets in the units of the upstream asset.

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants