Local Kubernetes development clusters: the 5 best local cluster compared

If you develop software that runs on Kubernetes, a local development cluster can significantly speed up your development iteration speed, because everything happens locally. I discuss the pros and cons of local clusters and then present the tools Docker for Desktop, Rancher Desktop, kind, k3d and Minikube, and how they compare. I finish with details regarding how to get locally-built images into the cluster, and how to get the Ingress to work.

Introduction

For us developers, the tooling is very important. It helps us to stay productive. Modern IDEs (such as Microsoft’s Visual Studio, VS Code, or Jetbrain’s IntelliJ-based IDEs) not only offer efficient code navigation, but they also let us easily attach a debugger, display logs, or inspect the contents of our local database, all without having to go through a complicated setup.

With the advancements of how software is deployed, the ways how we develop software also needed to change, to stay (somewhat) “in sync” with how software runs in production. Some 10-15 years ago, we used to copy compiled application bundles (such as jar files or other binaries) onto production servers. Development happened “natively” on our development host OS, where we installed compilers, language runtimes and frameworks. Then the shift to (Docker) containers and images happened, with IDEs adapting as well, giving us the ability to develop inside a container. With Kubernetes having become “mainstream” since a few years for many projects, developers need to adapt again, learning new tooling. As you can also see in the Development support tools article, IDEs alone no longer suffice – additional (external) tools are needed.

One such “tool” is a development Kubernetes cluster, which is now a basic requirement for developing Kubernetes-based applications. This article presents several alternatives for locally-installable Kubernetes clusters. Installing them is quite easy, as they all offer streamlined installation routines.

Why use a local Kubernetes development cluster?

Many developers I talked to use remote development clusters, often set up by their organization’s platform team. So why should you consider using a local cluster instead? Let’s look at the pros and cons:

Clusters

Let’s take a look at the most streamlined and popular local clusters.

Docker for Desktop

Docker for Desktop comes with a simple, single-node Kubernetes cluster that you can start or stop from the Docker for Desktop GUI.

Rancher Desktop

Rancher Desktop is a free and open source replacement for Docker for Desktop, available on Windows, macOS and Linux. It includes two container runtimes (moby and containerd). Rancher Desktop offers a single-node Kubernetes cluster. It comes with a simple-to-use installer and a simple GUI.

kind

kind is a lightweight CLI-based tool that runs single- or multi-node clusters. It requires a Docker(-compatible) engine installed on your host.

k3d

The underlying technology of k3d is k3s. k3s is a lightweight, production-grade Kubernetes distro which you could run directly on your host if you are on Linux. k3d is a cross-platform CLI wrapper around k3s, which lets you easily run single- or multi-node k3s clusters. It requires a Docker(-compatible) engine installed on your host. People have reported that k3d also works with Podman v4 or newer, or with Rancher Desktop with the Moby runtime, with some smaller caveats.

Minikube

Minikube is a CLI tool that runs single- or multi-node clusters on top of various virtualization technologies, such as VMs or Docker.

Comparison table

	Docker for Desktop	Rancher Desktop	kind	k3d	Minikube
Supports multi-node clusters	❌	❌	✅	✅	✅
No copying of locally-built images into cluster is required (faster)	✅	✅	❌	❌	❌
Choosable Kubernetes version	❌	✅	✅	✅	✅
Works in CI	❌	❌	✅	✅	✅
Built-in load balancer (for multi-node setups)	not applicable	not applicable	❌	✅	✅
Requirements	Desktop OS	Desktop OS	Docker daemon socket	Docker daemon socket	Docker daemon socket, or hypervisor

Getting images into the cluster

To test your local code changes, your cluster must be able to access the (locally-built) Docker/OCI images that contain your code. There are several ways to achieve this:

• Push the image to a remote registry: you have to configure the registry’s credentials in your local cluster, so that it can pull images
• Push the image to a local registry: a local registry is usually not access-protected, but it must be accessible (TCP/network) from both the (Docker) build engine (when pushing) and from within the cluster (pulling)
• Load the image into the cluster’s container runtime directly (in case the cluster uses a different container runtime than the one you used to build the image) – kind, k3d and minikube offer shell commands to load the image into the cluster’s container runtime

Performance-wise, the third approach is often slower than the other two approaches. Since pushing to a local registry is faster than pushing to a remote registry, this makes the second approach the most performant one (at increased setup costs).

The third approach has another caveat: it only works if the container runtime does not attempt to pull the image. Pulling it would fail, because pulling always requires a registry (that is, the images cached locally by the container runtime are ignored). Thus, if your Kubernetes workload objects (e.g. Pods) specify a imagePullPolicy: Always, it won’t work. It also won’t work if your workload object does not specify an imagePullPolicy at all: in this case, Kubernetes uses the default value, IfNotPresent. That sounds good (“use cached images if available”), but if your referenced images has the “:latest” version tag, this triggers special behavior where Kubernetes attempts to pull the image regardless of the imagePullPolicy, ignoring the IfNotPresent default value (see docs).

Getting Ingress to work

Often you want to access HTTP-based services from the browser running on your host. Although you can use kubectl port-forward to punch holes into your cluster, you may also want to use the ingress of your local cluster, which is typically something like an Nginx or Traefik that comes with a Deployment and a Service of type LoadBalancer. The instructions of how to get this working depend on the local cluster you are using:

• Docker for Desktop: because Docker for Desktop automatically publishes ports for services of type LoadBalancer on the host, following the official instructions (e.g. for Nginx see here) will only work if the ports (typically 80 and 443) are not already in use on your host. At least on Windows, however, it is very likely that such ports are already in use. The work around (for Nginx) is to simply download the deploy.yaml file mentioned in the official Nginx instructions, and modify the the ports (80 and 443) to something that is not yet in use on your host (see also here), then kubectl apply your locally-modified copy of the yaml file
• Rancher Desktop: services of type LoadBalancer (such as the Ingress) are automatically given an IP reachable from the host. If necessary, you can configure port mappings in the GUI of Rancher Desktop.
• kind: use kind’s extraPortMapping config option (docs)
• k3d: As explained in the official docs here, simply add the following to your k3d cluster create command: -p "8081:80@loadbalancer". You can now access your ingress on port 8081 on the host
• minikube: use minikube tunnel, which establishes a temporary load balancer, providing an external IP to Services that are of type: LoadBalancer (whose external IP would otherwise remain pending)

Conclusion

Assuming you have sufficient memory, local Kubernetes clusters can significantly speed up your development iteration speed, because everything happens locally. To choose the right one, you need to consider your requirements, such as:

• Do you need a multi-node cluster? If not, Rancher Desktop or Docker for Desktop are the fastest option, because you do not have to push locally-built images into the cluster
• Do you need to control the Kubernetes version? If so, Docker for Desktop is not suitable.
• Do you want to use the same test cluster in development and in CI? This would allow you to run automated CI tests locally as well.
• Which development tools (such as Tilt or DevSpace) do you want to use? How well do they support your chosen local cluster? I present these tools in part 3 of this article series.

Which local cluster do you use? Let me know in the comments!