Karpenter: Add the EC2 Karpenter agent #803
Conversation
ecpullen
force-pushed
the
karpenter
branch
2 times, most recently
from
95d65a4
to
0651a00
Compare
| pub struct Ec2KarpenterDestroyer {} | ||
|
|
||
| #[async_trait::async_trait] | ||
| impl Destroy for Ec2KarpenterDestroyer { |
There was a problem hiding this comment.
Does this clean up everything? Or does it leave some things behind?
There was a problem hiding this comment.
It cleans up all of the conflicting things. You can run this agent with the regular ec2 agent and everything works. In the future work section (which will become new issues) I've included improving cleanup.
| @@ -77,6 +77,7 @@ where | |||
| .args(k8s_image_arg) | |||
| .args(e2e_repo_arg) | |||
| .args(sonobuoy_image_arg) | |||
| .arg("--plugin-env=e2e.E2E_EXTRA_ARGS=--non-blocking-taints=sonobuoy") | |||
There was a problem hiding this comment.
For normal sonobuoy testing, nothing. For karpenter testing it allows nodes in the cluster to have the NoSchedule taint effect. The taint prevents sonobuoy nodes from being scheduled on the AL tainted nodegroup nodes.
Dockerfile
Outdated
| RUN yum install -y git make tar \ | ||
| && curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 \ | ||
| && chmod +x get_helm.sh && ./get_helm.sh --version v3.8.2 | ||
| # Copy eksctl |
There was a problem hiding this comment.
We should check a sha, or create an issue where we need to check shas for all of the things we download from various places in this dockerfile.
| RUN yum install -y git make tar \ | |
| && curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 \ | |
| && chmod +x get_helm.sh && ./get_helm.sh --version v3.8.2 | |
| # Copy eksctl | |
| RUN yum install -y git make tar \ | |
| && curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 \ | |
| && chmod +x get_helm.sh && ./get_helm.sh --version v3.8.2 | |
| # Copy eksctl |
There was a problem hiding this comment.
I am working on fixing that.
ecpullen
force-pushed
the
karpenter
branch
2 times, most recently
from
f6a0ac2
to
13b4a27
Compare
ecpullen
force-pushed
the
karpenter
branch
3 times, most recently
from
ad825c6
to
c118c1c
Compare
Dockerfile
Outdated
| # Builds the EC2 karpenter resource agent image | ||
| FROM public.ecr.aws/amazonlinux/amazonlinux:2 as ec2-karpenter-resource-agent | ||
|
|
||
| RUN yum install -y git make tar |
There was a problem hiding this comment.
If you do an update, you can keep the image size a little smaller if you add && yum -y clean all && rm -fr /var/cache to the end here.
There was a problem hiding this comment.
| RUN yum install -y git make tar | |
| RUN yum install -y git-core make tar |
You can also save some space with the git core sub-package.
ecpullen
force-pushed
the
karpenter
branch
4 times, most recently
from
e081caf
to
2e0f246
Compare
Issue number:
N/A
Description of changes:
Adds support for launching nodes with karpenter.
Agent operations (creation):
KarpenterControllerRole-<CLUSTER-NAME>aws-authConfigMap of the clusterNoScheduleto prevent sonobuoy from using those nodesAgent operations (destruction):
Future Work:
Support custom block sizeseksctlcallskubectlfor creating the provisionerAddhelmto testsys-toolsTesting done:
Patched the version of TestSys used in the
bottlerocketmonorepo and rancargo make testusing the new karpenter agent as the bottlerocket provider.Logs
By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.