Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use the execWf spec to determine artifactgc strategy #10066

Merged
merged 9 commits into from Nov 22, 2022

Conversation

juliev0
Copy link
Contributor

@juliev0 juliev0 commented Nov 19, 2022

Signed-off-by: Julie Vogelmani julie_vogelman@intuit.com

Fixes #10015

This fixes the case where the entire Workflow is based on a WorkflowTemplate and the ArtifactGC strategy is on the Workflow level in the WorkflowTemplate.

The issue was that when searching for artifacts we were looking at woc.wf rather than woc.execWf (which includes the merged WorkflowTemplate).

Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
}
}
return results
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use woc.execWf to get the strategy defined in the WorkflowTemplate if there is one

Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
spec:
artifactGC:
strategy: OnWorkflowDeletion
workflowMetadata:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unlike the WorkflowTemplate test that was already in there, this one includes a strategy specified on the Workflow level

@@ -151,16 +147,16 @@ func (woc *wfOperationCtx) processArtifactGCStrategy(ctx context.Context, strate
}
templateName := node.TemplateName
if templateName == "" && node.GetTemplateRef() != nil {
templateName = node.GetTemplateRef().Name
templateName = node.GetTemplateRef().Template
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was accidentally referring to the WorkflowTemplate's Name here rather than the name of the template within the WorkflowTemplate

Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
Signed-off-by: Julie Vogelmani <julie_vogelman@intuit.com>
if template == nil {
return fmt.Errorf("can't process Artifact GC Strategy %s: template name %q belonging to node %+v not found??", strategy, node.TemplateName, node)
return fmt.Errorf("can't process Artifact GC Strategy %s: template name %q belonging to node %+v not found??", strategy, templateName, node)
Copy link
Contributor Author

@juliev0 juliev0 Nov 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I confirmed that I no longer see any more cases of woc.wf used in this file that should really be woc.execWf

@juliev0 juliev0 marked this pull request as ready for review November 20, 2022 00:31
@@ -104,11 +104,7 @@ func (woc *wfOperationCtx) processArtifactGCStrategy(ctx context.Context, strate
woc.log.Debugf("processing Artifact GC Strategy %s", strategy)

// Search for artifacts
artifactSearchResults := woc.wf.SearchArtifacts(&wfv1.ArtifactSearchQuery{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just change to wow.execWF.SearchArtifacts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried that at first but the execWf doesn’t seem to include the Workflow Status (for node status). Should it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're correct, you must inspect woc.execWF.Spec and woc.wf.Status. The former is what to run, the latter what to update at the end of reconciliation.

You should not query across the two.

Copy link
Contributor Author

@juliev0 juliev0 Nov 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this look okay then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juliev0 Make sure you include the test for workflowdefautls, workflow template

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sarabala1979 I just ran a couple of tests locally to test WorkflowDefaults. In one, I just used a Workflow by itself and verified that it took the default from WorkflowDefaults. In the second, I used a WorkflowTemplate as well. As discussed, since the workflow-controller-configmap is not being dynamically reloaded, it's difficult to test this as part of the e2e test, since otherwise I need to change test/e2e/manifests/mixins/workflow-controller-configmap.yaml, which could have consequences for other e2e tests.

@sarabala1979 sarabala1979 enabled auto-merge (squash) November 22, 2022 20:07
@sarabala1979 sarabala1979 merged commit d03f5e5 into argoproj:master Nov 22, 2022
@juliev0 juliev0 deleted the 10015-artifactgctemplate branch November 22, 2022 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

artifactGC doesn't remove artifacts if defined in WorkflowTemplate
3 participants