This article explores my Kubernetes operator that provisions Azure Pipelines agents as Kubernetes Pods. It comes with an in-depth tutorial for how to use my operator. I also explore all alternative options for running Azure Pipeline agents, and explain why using a Kubernetes-based option beats all other options. Introduction CI/CD pipelines are a cornerstone to … Read more
This article explains BuildKit caching in detail. First you learn background knowledge, e.g. how the local and remote cache differs, and what kinds of cache items BuildKit can store where. Then, BuildKit’s garbage collection is explained in detail. Finally, I present 5 tricks to speed up your image builds, e.g. tuning the garbage collection policies, … Read more
This article explains how Docker builds images with BuildKit, via buildx. It discusses the advantages of BuildKit over Docker’s legacy image builder, and then elaborates on three approaches for calling BuildKit in Docker: 1) using the embedded BuildKit library managed by the Docker daemon, 2) using a BuildKit container managed by buildx, and 3) using … Read more
This article explains how to use “tar” correctly, to backup Docker volumes and restore them. I explain why two top-ranked tutorials are not doing a good job, by taking them apart. Finally, I give hints for creating backups of Docker volumes in production. Introduction Making a local backup of a Docker volume and restoring it … Read more
In this article I present 6 kinds of real-world Docker portability issues, where Docker-based software does not run or build the same way on two different machines. Introduction to Docker portability When I started learning about Docker several years ago, I was very enthusiastic about its promise of portability: write your software and Dockerfile once, … Read more
In this article I present two tools that help you with a Docker image diff or Docker image analysis: “dive” and the slim.ai Docker Desktop extension. I explain use cases for which you need these tools, and I show screen shots that illustrate their usage. Introduction Docker images (or more generally: OCI images) have become … Read more
The Docker build cache avoids rebuilding those parts of a Docker image that were already built. Unfortunately, cache misses are hard to debug. In this article I explain three frequent yet unexpected reasons for cache misses, with solutions. One of them is that COPY or ADD statements are rebuilt, because files have changed. To diagnose … Read more
This article demonstrates how you can do infrastructure testing for Ansible roles and playbooks. I explain how the tools Vagrant and Molecule+Docker let you easily provision temporary VMs or Docker containers in which you can experimentally run your Ansible roles/playbooks, or even run unit testing in Continuous Integration. Introduction to infrastructure testing Ansible is a … Read more
In this article I discuss container-based development, where you not only run but also develop and debug software inside a locally-running Docker container. I explain the advantages and disadvantages of this approach, and look at VS Code’s development container feature, various features of IntelliJ-based IDEs, and how to implement an IDE-independent approach. Introduction to container-based … Read more
Using a Node.js example project I demonstrate how Docker-based caching can speed up your GitLab CI/CD pipelines even more than GitLab’s built-in caching mechanism. I explain how each approach works, and what the technical prerequisites are. I also list tools that support you with setting up a Docker-based CI pipeline. Introduction Caching of files between … Read more