[WIP] ✨ Generation of typed apply clients using upstream generator #818

JoelSpeed · 2023-05-19T08:34:04Z

This integrates the upstream applyconfig generator into controller-tools to allow generation of ApplyConfig style types from custom API types.

The aim here is to provide the pointer style structs that could then be passed to an Apply style method for using server side apply.

Setting this up as a WIP PR for now, current TODO list:

Work out if there's any extra stuff here that's not needed
Work out how to test this without the hack adding cronjob testdata to the go.mod
Do we need OpenAPI schema/do we need the extract functions?
General review of code and cleanup - this was quite hacky when I started the branch
How would a user want this integrated into their project?

k8s-ci-robot · 2023-05-19T08:34:06Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

JoelSpeed · 2023-05-19T09:43:16Z

pkg/applyconfigurations/gen.go

+	"k8s.io/api/":                   "k8s.io/client-go/applyconfigurations/",
+}
+
+const importPathSuffix = "applyconfiguration"


Should this be configurable via a marker?

This is now available to be customized with the marker kubebuilder:ac:output:package

JoelSpeed · 2023-05-19T09:44:24Z

pkg/applyconfigurations/gen.go

+	enableTypeMarker = markers.Must(markers.MakeDefinition("kubebuilder:ac:generate", markers.DescribesType, false))
+)
+
+var importMapping = map[string]string{


Should we allow additional import mappings, I know the upstream applyconfiguration gen does

I think that makes sense, yeah (But we can do that in a follow-up)

JoelSpeed · 2023-05-19T09:46:27Z

/test all

Need to work out how to test this, but want to see what the existing tests do with this so I can fix any failures introduced

JoelSpeed · 2023-05-19T15:39:01Z

/test all

JoelSpeed · 2023-05-19T15:52:25Z

pkg/applyconfigurations/applyconfiguration_integration_test.go

+		filesInMaster := make(map[string][]byte)
+		masterFileNames := sets.New[string]()
+		cronJobFS := os.DirFS(".")
+		masterPath := "applyconfiguration-master"


Because I can't change the directory to which we are generating the assets currently, this is what I came up with, create a copy that we keep checked in and then diff against that, if there are genuine changes we can update both copies.

Will look at gengo to see if there's a backwards compatible way to allow it to output to different places like OutputRule does for these generators

JoelSpeed · 2023-05-19T15:56:13Z

/assign @alvaroaleman

EraYaN · 2023-05-22T07:49:50Z

As an option I would love to have the Extract functions, we use them extensively in our operator (for better or for worse).

sbueringer · 2023-05-23T09:04:15Z

pkg/applyconfigurations/gen.go

+	genericArgs, _ := generatorargs.NewDefaults()
+	genericArgs.InputDirs = []string{root.PkgPath}
+	genericArgs.OutputPackagePath = filepath.Join(root.PkgPath, importPathSuffix)
+	genericArgs.GoHeaderFilePath = ctx.HeaderFilePath


If I see correctly this currently stores the generated files somewhere below GOPATH/src (which isn't where I have the controller-tools repo). Would be probably good to align this to how we can set output dir in other generators.

I tried to run this in Cluster API with ~ apply paths=./api/... I end up with files generated in $GOPATH/src/sigs.k8s.io/cluster-api/api/v1beta1/applyconfiguration/api/v1beta1

Not sure if it's good if we store the applyconfigurations in a sub-package of the API types. Might be a simple output:apply:dir would be better and then we can store it below there, e.g.

output:apply:dir is explicitly set to $GOPATH/src/sigs.k8s.io/cluster-api/applyconfigurations

files are then generated in e.g. $GOPATH/src/sigs.k8s.io/cluster-api/applyconfiguration/api/v1beta1

I think this would follow the same pattern as the ones in client-go. (side note: the API folders are not well structured in Cluster API)

(probably this is covered by #818 (comment) feel free to resolve)

sbueringer · 2023-05-23T09:29:10Z

@JoelSpeed Can we get rid of the vendor folder? It's probably on my side but the performance of the file tab is pretty bad with 1700 files.

sbueringer · 2023-05-23T09:37:25Z

pkg/applyconfigurations/testdata/cronjob/applyconfiguration-master/internal/internal.go

@@ -0,0 +1,46 @@
+// Code generated by applyconfiguration-gen. DO NOT EDIT.
+
+package internal


Q: Why this is generated?

applyconfiguration-gen always outputs this. The Parser() item is used in the internal extract implementations. And that schema yaml defines what items can then be parsed I believe.

called like this in extractCrdType

internal.Parser().Type("com.domain.group.api.v1.CrdType")

alvaroaleman · 2023-05-23T20:55:23Z

@JoelSpeed Can we get rid of the vendor folder? It's probably on my side but the performance of the file tab is pretty bad with 1700 files.

Second this, I can literally not review this through the GitHub UI

JoelSpeed · 2023-05-26T14:23:47Z

@JoelSpeed Can we get rid of the vendor folder? It's probably on my side but the performance of the file tab is pretty bad with 1700 files.

Second this, I can literally not review this through the GitHub UI

Sorry about that, must have committed during my experimenting, dropped it now, commit history in need of a good clean though

alvaroaleman

For an internal project, I added the +kubebuilder:ac:generate=true comment both on the package and on my type, running controller-gen with apply paths=./pkg/api/v1/ succeeds but doesn't generate anything - any idea what I am am doing wrong?

alvaroaleman · 2023-05-23T20:49:02Z

pkg/applyconfigurations/gen.go

+	enableTypeMarker = markers.Must(markers.MakeDefinition("kubebuilder:ac:generate", markers.DescribesType, false))
+)
+
+var importMapping = map[string]string{


I think that makes sense, yeah (But we can do that in a follow-up)

alvaroaleman

For an internal project, I added the +kubebuilder:ac:generate=true comment both on the package and on my type, running controller-gen with apply paths=./pkg/api/v1/ succeeds but doesn't generate anything - any idea what I am am doing wrong?

sbueringer · 2023-05-30T17:49:57Z

For an internal project, I added the +kubebuilder:ac:generate=true comment both on the package and on my type, running controller-gen with apply paths=./pkg/api/v1/ succeeds but doesn't generate anything - any idea what I am am doing wrong?

Potentially could be: #818 (comment)

k8s-ci-robot · 2023-09-18T09:18:50Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: JoelSpeed
Once this PR has been reviewed and has the lgtm label, please ask for approval from alvaroaleman. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

JoelSpeed · 2023-09-18T09:20:39Z

@jakobmoellerdev Reached out to me last week and sent me a patch that we think resolves a number of the comments here.

@jakobmoellerdev Anything the reviewers need to know about what you've changed?

jakobmoellerdev · 2023-09-18T09:21:50Z

pkg/applyconfigurations/gen.go

+	"k8s.io/api/":                   "k8s.io/client-go/applyconfigurations/",
+}
+
+const importPathSuffix = "applyconfiguration"


This is now available to be customized with the marker kubebuilder:ac:output:package

jakobmoellerdev · 2023-09-18T09:24:35Z

pkg/applyconfigurations/gen.go

+	// looking into the package and retrieving the first go file it finds, and using that as the output base.
+	// this is because we cannot rely on gogen calculating the correct output base.
+	// if we leave this empty, gogen will attempt to use GOPATH to write the files which is not wanted
+	genericArgs.OutputBase = filepath.Dir(root.GoFiles[0])


I think with some more invest it may be possible to resolve the directory smoother but there we issues with the output package not having correct import paths. For now this will work fine with the generation of the base directory being in the package path. For the request of having a directory outside of the package path this needs adjustment though. Open to any ideas here

I believe in general it should now work both within and outside GOPATH and should resolve the issue @sbueringer and @alvaroaleman not seeing the generated files in the right folders

jakobmoellerdev · 2023-09-18T09:27:10Z

pkg/oot_testsubdirectory

I believe this file was included due to the patch, should be removed from the final version

k8s-ci-robot · 2023-10-20T02:36:35Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

erikgb · 2023-10-21T10:06:47Z

@JoelSpeed Do you have some bandwidth to follow up on this PR? I just added the upstream generator to one of our projects to get some progress towards SSA, and it was NOT a pleasant experience! I would love to see the ability to generate apply clients with controller-gen.

JoelSpeed · 2023-10-23T09:14:37Z

@erikgb I'm currently waiting on a second round of review from those who previously reviewed. @jakobmoellerdev kindly continued the work and we pushed his updates to my branch, but haven't had a review since then

jakobmoellerdev · 2024-01-18T18:06:45Z

New year new bump :) any chance this can get another round of review? I can resolve the merge conflict if necessary @JoelSpeed, please let me know if I should take another look

k8s-triage-robot · 2024-04-17T18:21:57Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

sbueringer · 2024-04-17T18:44:01Z

/remove-lifecycle stale
/lifecycle frozen

Apologies for the further delay. Really trying to get around to it, just too much going on.

k8s-ci-robot · 2024-04-17T18:44:03Z

@sbueringer: The lifecycle/frozen label cannot be applied to Pull Requests.

In response to this:

/remove-lifecycle stale
/lifecycle frozen

Apologies for the further delay. Really trying to get around to it, just too much going on.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 19, 2023

k8s-ci-robot requested review from joelanford and vincepri May 19, 2023 08:34

k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label May 19, 2023

JoelSpeed force-pushed the applyconfig-gen-integration branch 2 times, most recently from 18594e4 to 74642c2 Compare May 19, 2023 09:38

JoelSpeed commented May 19, 2023

View reviewed changes

JoelSpeed mentioned this pull request May 19, 2023

✨ Generation of typed apply clients #536

Open

JoelSpeed marked this pull request as ready for review May 19, 2023 15:46

JoelSpeed commented May 19, 2023

View reviewed changes

k8s-ci-robot assigned alvaroaleman May 19, 2023

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 22, 2023

sbueringer reviewed May 23, 2023

View reviewed changes

alvaroaleman reviewed May 30, 2023

View reviewed changes

joelanford mentioned this pull request Jun 8, 2023

add support for handling plain+v0 bundle types operator-framework/operator-controller#242

Merged

4 tasks

yujunz mentioned this pull request Jul 27, 2023

Opt-in to tolerate ad-hoc modification in resources alexandrevilain/temporal-operator#418

Open

jakobmoellerdev mentioned this pull request Aug 23, 2023

OCPVE-613: feat: switch to SSA for resource reconciliation openshift/lvm-operator#395

Closed

5 tasks

Stub out generator

efb1b04

JoelSpeed and others added 11 commits September 15, 2023 15:41

Add generated clients

1f4108c

Remove makefile based generator for applyconfigs

73e50fc

Cleanup unused code from previous implementation

ef7e022

Move generated applyconfiguration to applyconfiguration folder from ac

e6f22fa

Dont need gitignore change

b54d13c

Cleanup markers, ensure correct path for non GOPATH

fd0849e

Drop util

1d525dc

Add test to check generated content against generated master

c75bd68

Fix linting issues

9346867

Drop vendor

49b2a52

chore: resolve applyconfig gen correctly

0aa6676

JoelSpeed force-pushed the applyconfig-gen-integration branch from 607e978 to 0aa6676 Compare September 18, 2023 09:18

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 18, 2023

jakobmoellerdev reviewed Sep 18, 2023

View reviewed changes

pkg/oot_testsubdirectory

Copy link

jakobmoellerdev Sep 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this file was included due to the patch, should be removed from the final version

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 20, 2023

This was referenced Oct 25, 2023

chore: generate applyconfigurations for API statnett/image-scanner-operator#668

Merged

chore: generate applyconfigurations for custom resources cert-manager/trust-manager#217

Merged

alvaroaleman mentioned this pull request Mar 8, 2024

⚠ Client Server side apply kubernetes-sigs/controller-runtime#2702

Draft

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 17, 2024

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 17, 2024

		@@ -0,0 +1,46 @@
		// Code generated by applyconfiguration-gen. DO NOT EDIT.

		package internal

[WIP] ✨ Generation of typed apply clients using upstream generator #818

Are you sure you want to change the base?

[WIP] ✨ Generation of typed apply clients using upstream generator #818

Conversation

JoelSpeed commented May 19, 2023 • edited

k8s-ci-robot commented May 19, 2023

Choose a reason for hiding this comment

jakobmoellerdev Sep 18, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoelSpeed commented May 19, 2023

JoelSpeed commented May 19, 2023

Choose a reason for hiding this comment

JoelSpeed commented May 19, 2023

EraYaN commented May 22, 2023

sbueringer May 23, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbueringer May 23, 2023 • edited

Choose a reason for hiding this comment

sbueringer May 23, 2023 • edited

Choose a reason for hiding this comment

sbueringer commented May 23, 2023

sbueringer May 23, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvaroaleman commented May 23, 2023

JoelSpeed commented May 26, 2023

alvaroaleman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvaroaleman left a comment

Choose a reason for hiding this comment

sbueringer commented May 30, 2023 • edited

k8s-ci-robot commented Sep 18, 2023

JoelSpeed commented Sep 18, 2023

jakobmoellerdev Sep 18, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Oct 20, 2023

erikgb commented Oct 21, 2023

JoelSpeed commented Oct 23, 2023

jakobmoellerdev commented Jan 18, 2024 • edited

k8s-triage-robot commented Apr 17, 2024

sbueringer commented Apr 17, 2024

k8s-ci-robot commented Apr 17, 2024

JoelSpeed commented May 19, 2023 •

edited

jakobmoellerdev Sep 18, 2023 •

edited

sbueringer May 23, 2023 •

edited

sbueringer May 23, 2023 •

edited

sbueringer May 23, 2023 •

edited

sbueringer May 23, 2023 •

edited

sbueringer commented May 30, 2023 •

edited

jakobmoellerdev Sep 18, 2023 •

edited

jakobmoellerdev commented Jan 18, 2024 •

edited