Azure Pipelines Kubernetes operator feature

Auto-scaling Azure Pipelines Agents in Kubernetes

This article explores my Kubernetes operator that provisions Azure Pipelines agents as Kubernetes Pods. It comes with an in-depth tutorial for how to use my operator. I also explore all alternative options for running Azure Pipeline agents, and explain why using a Kubernetes-based option beats all other options. Introduction CI/CD pipelines are a cornerstone to … Read more

BuildKit caching feature

Advanced Docker / BuildKit Caching with 5 tricks to speed up your image builds

This article explains BuildKit caching in detail. First you learn background knowledge, e.g. how the local and remote cache differs, and what kinds of cache items BuildKit can store where. Then, BuildKit’s garbage collection is explained in detail. Finally, I present 5 tricks to speed up your image builds, e.g. tuning the garbage collection policies, … Read more

Docker and BuildKit feature

Docker and BuildKit: how building images really works

This article explains how Docker builds images with BuildKit, via buildx. It discusses the advantages of BuildKit over Docker’s legacy image builder, and then elaborates on three approaches for calling BuildKit in Docker: 1) using the embedded BuildKit library managed by the Docker daemon, 2) using a BuildKit container managed by buildx, and 3) using … Read more

back up Docker volumes feature image

Backup Docker volumes (and restore them) – done right

This article explains how to use “tar” correctly, to backup Docker volumes and restore them. I explain why two top-ranked tutorials are not doing a good job, by taking them apart. Finally, I give hints for creating backups of Docker volumes in production. Introduction Making a local backup of a Docker volume and restoring it … Read more

Docker portability issues feature

Docker portability: 6 important caveats and pitfalls

In this article I present 6 kinds of real-world Docker portability issues, where Docker-based software does not run or build the same way on two different machines. Introduction to Docker portability When I started learning about Docker several years ago, I was very enthusiastic about its promise of portability: write your software and Dockerfile once, … Read more

Docker image analysis and Docker image diff feature

Docker image analysis and diffing: what, how and why

In this article I present two tools that help you with a Docker image diff or Docker image analysis: “dive” and the Docker Desktop extension. I explain use cases for which you need these tools, and I show screen shots that illustrate their usage. Introduction Docker images (or more generally: OCI images) have become … Read more

Docker build cache debug techniques

Docker build cache: debug techniques

The Docker build cache avoids rebuilding those parts of a Docker image that were already built. Unfortunately, cache misses are hard to debug. In this article I explain three frequent yet unexpected reasons for cache misses, with solutions. One of them is that COPY or ADD statements are rebuilt, because files have changed. To diagnose … Read more

infrastructure testing for Ansible roles and playbooks feature

Infrastructure testing for Ansible playbooks and roles: an introductory guide

This article demonstrates how you can do infrastructure testing for Ansible roles and playbooks. I explain how the tools Vagrant and Molecule+Docker let you easily provision temporary VMs or Docker containers in which you can experimentally run your Ansible roles/playbooks, or even run unit testing in Continuous Integration. Introduction to infrastructure testing Ansible is a … Read more

container-based development environments feature

Container-based development environments

In this article I discuss container-based development, where you not only run but also develop and debug software inside a locally-running Docker container. I explain the advantages and disadvantages of this approach, and look at VS Code’s development container feature, various features of IntelliJ-based IDEs, and how to implement an IDE-independent approach. Introduction to container-based … Read more

gitlab vs docker caching

GitLab CI/CD: GitLab vs. Docker caching – how to get insanely fast CI/CD pipelines

Using a Node.js example project I demonstrate how Docker-based caching can speed up your GitLab CI/CD pipelines even more than GitLab’s built-in caching mechanism. I explain how each approach works, and what the technical prerequisites are. I also list tools that support you with setting up a Docker-based CI pipeline. Introduction Caching of files between … Read more