[DS][40/n] Create an as_auto_materialize_policy method on SchedulingCondition

[OwenKephart]

Summary & Motivation

As title -- this is a more convenient way of converting a condition to a policy, as it doesn't require an additional import.

Also makes sure that the policy survives serdes and all that

How I Tested These Changes

[OwenKephart]

• [DS][40/n] Create an as_auto_materialize_policy method on SchedulingCondition #21788 👈
• [DS][39/n] Use FailedSchedulingCondition and ScheduledSince condition in eager policy #21741
• [DS][38/n] Create FailedSchedulingCondition #21740
• [DS][37/n] Reorganize AssetGraphView methods #21739
• [DS][36/n] Create ScheduledSince condition #21712
• [DS][35/n] Replace asset_subsets_with_metadata with asset_slices_with_metadata #21793
• [DS][34/n] Remove some AssetCondition references #21704
• [DS][33/n] Create static constructor for cron condition #21703
• [DS][32/n] Update UpdatedSinceCronCondition #21677
• [DS][31/n] Create static constructor for eager condition #21737
• [DS][30/n] Create new unique ids for each child based off of index #21671 : 1 other dependent PR (#21705 )
• [DS][29/n] Create ParentNewerCondition #21641 : 1 other dependent PR (#21670 )
• [DS][28/n] Create WillBeRequestedCondition #21640
• [DS][27/n] Make candidate_slice and true_slice Optional #21648
• [DS][26/n] Update internal representation of InLatestTimeWindow #21615
• [DS][25/n] Narrow extra_state type #21613
• [DS][24/n] Remove all AssetSubset references from the context and result objects #21612
• [DS][23/n] Update SchedulingContext object #21573
• [DS][22/n] Convert SchedulingResult to DagsterModel #21546
• [DS][21/n] Rename AssetConditionResult to SchedulingResult #21545
• [DS][20/n] Port static constructors from AssetCondition to SchedulingCondition #21541
• [DS][19/n] Create SchedulingCondition object to replace AssetCondition #21540
• [DS][18/n] Move serialized objects into separate file #21539
• [DS][17/n] Move legacy files to separate legacy folder #21538
• [DS][16/n] Rename AssetConditionEvaluationContext to LegacyRuleEvaluationContext #21537
• [DS][15/n] Move operators to separate folder #21536
• [DS][14/n] Remove unused methods and properties #21535
• [DS][13/n] Convert SchedulingContext to DagsterModel #21521
• [DS][12/n] Rename SchedulingContext object #21520
• [DS][11/n] Create InProgressCondition #21511 : 1 other dependent PR (#21512 )
• [DS][10/n] Create UpdatedSinceCronCondition #21510
• [DS][9/n] Add lookback_timedelta to InLatestTimeWindow condition #21508
• [DS][8/n] Create InLatestTimeWindow condition #21507
• [DS][7/n] Create MaterializedSchedulingCondition #21505
• [DS][6/n] Create Any/AllDepsMatchSchedulingConditions #21504
• [DS][5/n] Use AssetSlice in AssetConditionResult #21503
• [DS][4/n] Rename asset_condition folder and add README #21502
• [DS][3/n] Update unique_id creation logic #21501
• [DS][2/n] Update signature of AutoMaterializeRule #21500
• [DS][1/n] Create SchedulingConditionEvaluationContext #21499
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @OwenKephart and the rest of your teammates on Graphite

[schrockn]

I think given this if I were a user the first think I would write is a shorthand function like as_amp_policy.

auto_materialize_policy=AutoMaterializePolicy.from_scheduling_condition(SchedulingCondition.eager() also an option

@asset(auto_materialize_policy=as_amp_policy(SchedulingCondition.eager())

I think the as_auto_materialize_policy might be a bit annoying to detail with in cases where there are complicated boolean experessions. I think it will be easy to make the mistake where you forgot to wrap the entire thing in parents.

[OwenKephart]

@schrockn "as amp policy" = "as auto materialize policy policy" (ATM machine-type thing). I think as_amp() might be too "informal".

AutoMaterializePolicy.from_scheduling_condition is possible for the user to do in this diff, but the drawback is they now need two imports (AutoMaterializePolicy and SchedulingCondition).

Considering we'd ideally want this construction to be fairly short-lived, I think a bit of verbosity isn't the end of the world, but open to pushback. Maybe the best solution is just to remove the as_auto_materialize_policy() thing and force users to go through from_scheduling_condition()?

[schrockn]

Question/alt suggestion inline

[schrockn]

I think I prefer AutoMaterializePolicy.from_scheduling_condition but we can change this once we get going writing docs etc so let's just press on

[OwenKephart]

Merge activity

• May 14, 10:38 AM EDT: @OwenKephart started a stack merge that includes this pull request via Graphite.
• May 14, 11:17 AM EDT: Graphite rebased this pull request as part of a merge.
• May 14, 11:18 AM EDT: @OwenKephart merged this pull request with Graphite.