Skip to content

Commit

Permalink
[SPARK-41502][K8S][TESTS] Upgrade the minimum Minikube version to 1.28.0
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to upgrade the minimum Minikube version to 1.18.0 to 1.28.0 for Apache Spark 3.4.0.

### Why are the changes needed?

Minikube v1.28.0 is released on `Nov 4th, 2022` and the latest one as of today.
- https://github.com/kubernetes/minikube/releases/tag/v1.28.0

GitHub Action CI has been using `Minikube 1.28.0` to test `K8s v1.25.3` and Homebrew also provides `1.28.0` by default.
- https://github.com/apache/spark/actions/runs/3681318787/jobs/6227888255
```
* minikube v1.28.0 on Ubuntu 20.04
...
* Downloading Kubernetes v1.25.3 preload ...
```

In addition, we can choose different K8s versions on this latest Minikube like the following.
```
$ minikube start --kubernetes-version=1.21.0
😄  minikube v1.28.0 on Darwin 13.1 (arm64)
❗  Kubernetes 1.21.0 has a known performance issue on cluster startup. It might take 2 to 3 minutes for a cluster to start.
❗  For more information, see: kubernetes/kubeadm#2395
✨  Automatically selected the docker driver
📌  Using Docker Desktop driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.21.0 preload ...
```

### Does this PR introduce _any_ user-facing change?

No. This is a dev-only change.

### How was this patch tested?

Pass the CIs.

Closes apache#39043 from dongjoon-hyun/SPARK-41502.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
dongjoon-hyun authored and beliefer committed Dec 18, 2022
1 parent 3d14fc9 commit a0a9c5f
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions resource-managers/kubernetes/integration-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ To run tests with Hadoop 2.x instead of Hadoop 3.x, use `--hadoop-profile`.

./dev/dev-run-integration-tests.sh --hadoop-profile hadoop-2

The minimum tested version of Minikube is 1.18.0. The kube-dns addon must be enabled. Minikube should
The minimum tested version of Minikube is 1.28.0. The kube-dns addon must be enabled. Minikube should
run with a minimum of 4 CPUs and 6G of memory:

minikube start --cpus 4 --memory 6144
Expand All @@ -46,7 +46,7 @@ default this is set to `minikube`, the available backends are their prerequisite

### `minikube`

Uses the local `minikube` cluster, this requires that `minikube` 1.18.0 or greater be installed and that it be allocated
Uses the local `minikube` cluster, this requires that `minikube` 1.28.0 or greater be installed and that it be allocated
at least 4 CPUs and 6GB memory (some users have reported success with as few as 3 CPUs and 4GB memory). The tests will
check if `minikube` is started and abort early if it isn't currently running.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ private[spark] object Minikube extends Logging {

versionArrayOpt match {
case Some(Array(x, y, z)) =>
if (Ordering.Tuple3[Int, Int, Int].lt((x, y, z), (1, 18, 0))) {
if (Ordering.Tuple3[Int, Int, Int].lt((x, y, z), (1, 28, 0))) {
assert(false, s"Unsupported Minikube version is detected: $minikubeVersionString." +
"For integration testing Minikube version 1.18.0 or greater is expected.")
"For integration testing Minikube version 1.28.0 or greater is expected.")
}
case _ =>
assert(false, s"Unexpected version format detected in `$minikubeVersionString`." +
Expand Down

0 comments on commit a0a9c5f

Please sign in to comment.