Persisting the running apps on KubeApps and starting and stopping apps

[SohrabTa]

Hello KubeApps community,

I want to pre-fill the values.yaml of a service via the Form or directly in the Yaml tab in KubeApps and start a service like this but also be able to stop it and reuse the same configuration of values.yaml the next day or next week. I was not able to find a way yet, besides deleting the App if I want to stop it.

If there is really no way for that yet:
Where does KubeApps store the specific values.yaml of the revisions one can rollback to? Or does KubeApps use the helmClient function rollback? And if yes, does someone know how and where helm stores those?

TL;DR
Is there a way to make specific configurations of values.yaml of the started helm charts in KubeApps persistent?

[absoludity]

Hi there @SohrabTa .

Regarding stopping and starting an installation: no, Kubeapps does not provide a way to do this via the UI, since it would depend on the specific installation (ie. it might require scaling one or more deployments back, or updating a custom resource etc.). Is that what you want to do, to leave a chart installed, but scale back it's resources so it's not consuming CPU etc?

Regarding multiple different values.yaml configurations to apply, see #859 for what may be similar discussions, but seems to have been thought that it is part of the chart (at least for standard prod/staging type values).

Kubeapps doesn't store the values.yaml but rather just passes this to helm to be handled by Helm. Yes, you can grab those for any installed chart via helm, see helm get values -h.

So TL;DR
You could have specific configuration of values.yaml to use with the Helm CLI when installing different charts, but we don't support that in the Kubeapps UI yet.

[SohrabTa]

Hello @absoludity!

thanks for the quick response.

Is that what you want to do, to leave a chart installed, but scale back it's resources so it's not consuming CPU etc?

Not exactly, I would like to have a failure safety system, that can recover the state of my running apps in the Kubernetes cluster in a disater case. The recovery system should pre-fill the values.yaml files of all the previously running apps and have them ready to go in the KubeApps Dashboard Applications tab. So an engineer can just reconfigure some parameter (if wanted) and start the app again.

Thanks a lot for the discussion link, this would be a solution for injecting pre-configured values.yaml, but as I can read it, it is not supported yet right?

And yes the helm get values function I had on the list as well, but in that case I'm afraid I'd have to manage a repository of values.yaml for all my apps, with version and user management. I will keep that solution in the back-hand though.

EDIT:
When I start my local minikube cluster, the helm charts deployed in a previous session are recovered and deployed already. How does this work? Where does KubeApps get the information to startup the Apps from the previous session? This would be a great indication for me dig into the persistency.

So TL;DR
My main goal is persistence of the apps' values.yaml from the most recent app revision and to be able to recover those in the error case (human or machine error).

[absoludity]

Not exactly, I would like to have a failure safety system, that can recover the state of my running apps in the Kubernetes cluster in a disater case. The recovery system should pre-fill the values.yaml files of all the previously running apps and have them ready to go in the KubeApps Dashboard Applications tab.

Kubeapps will display your helm releases from the cluster (no extra storage required). Thanks for the clarification, though it sounds like you're after a general disaster recovery system for K8s, not a feature of Kubeapps (or helm)? I mean, if you ran something like Velero, it would backup the helm releases (they're just k8s secrets) together with the values (same secret) as well as all the resources running in the cluster, which could be restored? I don't (yet) see why you think this functionality should be part of Kubeapps (which doesn't store any extra data for helm releases) and Helm (which just templates out a bunch of k8s resources).

So an engineer can just reconfigure some parameter (if wanted) and start the app again.

So the engineer could grab the backup from velero (or whatever backup solution), get the latest Helm release secret and grab the values from there, or could restore the running services directly from the backed up resources.

Thanks a lot for the discussion link, this would be a solution for injecting pre-configured values.yaml, but as I can read it, it is not supported yet right?

Correct (sorry, thought that was what you were asking for at first).

And yes the helm get values function I had on the list as well, but in that case I'm afraid I'd have to manage a repository of values.yaml for all my apps, with version and user management. I will keep that solution in the back-hand though.

Yeah, as opposed to letting a backup system do that for you.

EDIT: When I start my local minikube cluster, the helm charts deployed in a previous session are recovered and deployed already. How does this work?

I assume all that's happening is that the state of the docker container was saved to disk when you stop the cluster? (See https://minikube.sigs.k8s.io/docs/commands/stop/ ).

Where does KubeApps get the information to startup the Apps from the previous session? This would be a great indication for me dig into the persistency.

I don't think what you saw is anything to do with Kubeapps, but rather just minikube stop keeping state for you.

So TL;DR My main goal is persistence of the apps' values.yaml from the most recent app revision and to be able to recover those in the error case (human or machine error).

As above, I'd be looking at cluster backup options (like Velero). Hope that helps!

[SohrabTa]

Got it, KubeApps does not support backup and restore. Thanks a lot for the Velero tip! Along that path I stumbled upon Kasten K10. That can be used as an backup infrastructure behind our Kubernetes cluster.

So the engineer could grab the backup from velero (or whatever backup solution), get the latest Helm release secret and grab the values from there, or could restore the running services directly from the backed up resources.

Exactly. But I also found another crude workaround for "stopping" and "starting" an app in Kubeapps:
If I set the replicaCount in the Helm charts' values.yaml to zero there will be no pods of that app running, but it will stay in the Kubernetes cluster, so it will stay deployed in the Kubeapps Dashboard.
This way, the user can stop an app as needed and on the next day restart it and - if needed - even reconfigure the app's parameters.

I am not very proficient in Go or else I would offer to add it as a feature - a "Start/Stop" button next to the "Upgrade", "Rollback" and "Delete" buttons, that implements that functionality for future users.

Thanks a lot for your help. I think I will use a combination of Kasten K10 for persistence outside of the K8s cluster and the hacky replicaCount method to keep apps in the Kubeapps dashboard.

[absoludity]

Got it, KubeApps does not support backup and restore. Thanks a lot for the Velero tip! Along that path I stumbled upon Kasten K10. That can be used as an backup infrastructure behind our Kubernetes cluster.

Great!

Exactly. But I also found another crude workaround for "stopping" and "starting" an app in Kubeapps: If I set the replicaCount in the Helm charts' values.yaml to zero there will be no pods of that app running, but it will stay in the Kubernetes cluster, so it will stay deployed in the Kubeapps Dashboard. This way, the user can stop an app as needed and on the next day restart it and - if needed - even reconfigure the app's parameters.

Yeah, that's along the lines of what I meant above about scaling the deployment down. I assume that services will 503 or similar, but maybe that's no issue in your case.

Thanks a lot for your help. I think I will use a combination of Kasten K10 for persistence outside of the K8s cluster and the hacky replicaCount method to keep apps in the Kubeapps dashboard.

Glad to help :) Just note, it's not exactly clear to me why you want the start stop functionality, but another possibility is that you can helm delete --keep-history which would fully remove the resources from the cluster, but keep the release history (with values used etc.).