Kubernetes cluster introspection – the 5 best tools to better understand your cluster

Cluster introspection tools help you better understand what is going on in your Kubernetes cluster, by presenting the workloads visually. In this article I present a taxonomy of available tools and briefly present the tools Lens, k9s, kubenav, Octant, Headlamp, Kubernetes dashboard, Skooner and Portainer. I conclude by giving general recommendations.

Introduction

A very common task for developers, operations and SREs is to introspect a Kubernetes cluster, e.g. to diagnose an unhealthy cluster. The goal is to find out which Pods, Deployments and other Kubernetes workloads there are, whether this matches your expected set of workloads, and diagnose unhealthy deployments.

A commonly used tool is to diagnose problems is kubectl, which interacts with the Kubernetes API server. However, typing commands is slow (especially if you don’t remember the exact syntax), and often the output is difficult to understand. Instead, you might want to use cluster introspection tools, which also communicate with the Kubernetes API server, but let you interact with the cluster visually, by presenting a graphical user interface (GUI).

I’m presenting the tools following this taxonomy:

• Client-side tools: tools that you download and run on your local device. There are several variants:
  • Terminal/CLI: tools that run in a terminal / shell. Concrete tool: k9s
  • Window-based: graphical, window-based tools. Concrete tools: Lens, kubenav
  • Mobile app: an app for widespread platforms such as iOS and Android. Concrete tool: kubenav
  • Web server: starts a local web server that offers a web interface. Concrete tools: Octant, Headlamp
• Server-side tools: tools that you install into the cluster. They offer a web interface to authorized users, which can be accessed e.g. via the cluster’s Ingress. Concrete tools: Kubernetes dashboard, Skooner, Portainer, kubenav

Client-side cluster introspection tools

Lens

Lens is a client-side, window-based tool with many useful features, such as:

• Management of multiple clusters
• Shows many resource types in a table (Pods, Service, etc.)
• Create a kubectl port-forward with a single click
• Access logs or get a shell into a running pod with one click
• Decode secrets with one click
• Ability to edit manifests (YAML) in place
• Creates pre-configured terminals/shells where kubectl is already set to use the correct kubectl version and context (which is very handy when working with multiple clusters)
• Handles Helm charts (listing, installing, removing, changing configuration)
• Visualizes selected Prometheus metrics (CPU, memory, network and disk I/O)

Take a look at this YouTube video for an introduction.

While Lens probably has the most features of all tools presented here, it does have a few downsides:

• Sometimes views don’t refresh properly, showing stale data
• Sometimes uses a lot of CPU
• Consumes a lot of memory (easily 500 MB and more)

k9s

k9s is a client-side CLI (Terminal) tool with many advantages, such as:

• Shows all kinds of resources in a table (Pods, Service, etc.),
• Ability to quickly access logs, create port-forwards, get a shell, attach to container
• Much more memory/CPU-efficient than Lens

Check out this video for an introduction.

Lesser known client-side tools

There are a few more client-side tools you can try, which have similar features, but don’t otherwise stand out. These are:

• Octant, which offers a web interface served from a local web server (take a look at this video for a quick tour)
• Headlamp, which is both a client- and server-side tool. It can be started as Desktop application, via a local web server (called “headless mode” of the Desktop app), or you can install it into a cluster (this video shows how it looks)
• kubenav, which comes as iOS and Android app, as well as a Desktop app, and it can also be installed into the cluster (see screenshots here). Beware that this tools seems to be in maintenance mode, with no commits since January 2022.

Server-side cluster introspection tools

There are a few options available.

Kubernetes dashboard

The “official” Kubernetes dashboard is the oldest project, and is still actively maintained. Check out this video for a quick demo.

Skooner

Skooner is quite similar to the official Kubernetes dashboard. In addition to token-based authentication, Skooner offers OIDC authentication as well. It also seems to offer better layouting on mobile devices. Its development seems to have slowed down, though.

Portainer

Portainer is well-known in the Docker (Swarm) community, but it also supports Kubernetes. Portainer has a distributed architecture, consisting of an agent and a server, which can be (but don’t have to be) deployed separately. This lets you manage multiple clusters (into which you installed the agents) from a single Portainer server. Note that some functionality is only available in Portainer’s paid business edition (see feature comparison page).

Conclusion

Kubernetes cluster introspection tools greatly simplify the life of developers, ops or SRE. The visual presentation of information makes it easier to understand what is going on, compared to using text-based (CLI) tools. Most of these tools also automatically refresh their views.

But which of the many tools should you choose?

The main advantage of client-side tools is that every developer has full freedom of choice. Each developer can use something else, if preferred. I recommend you simply try Lens and k9s to see which tool suits you better. I found k9s to have a bit of a learning curve and initially requires a cheat sheet for the keyboard shortcuts. The main disadvantage of client-side tools is that every end-user of the tool must be technically able to access the cluster (from a networking perspective), which might not always be the case. For instance, developers might not be allowed to connect to the production cluster, only ops or SREs are.

On the other hand, server-side tools make sense if you need their main advantage: accessibility to everyone in your team. Other than that, the overhead of setting up server-side tools is larger than using client-side tools.

Finally, there are other server-side tools worth checking out, which primarily have a different purpose, but offer a nice introspection GUI as a side-effect. Examples are:

• Pixie: it uses eBPF to capture telemetry data, augmenting the observability of your cluster
• Kiali: a web UI for the Istio service mesh
• ArgoCD: a Kubernetes controller for GitOps, which also offers a comprehensive web UI. I have presented ArgoCD in more detail in this article

Which cluster introspection tools do you use? Let me know in the comments!