This article introduces several tricks that you can apply at build-time, to reduce the size of your Docker images, including the use of a small base image, multi-stage builds, consolidation of RUN statements, avoiding separate chown/chmod commands, or using the slim toolkit. Originally posted on 2022-02-06, updated on 2024-06-11. Introduction Docker has become a commodity … Read more
This article offers several tips for tweaking the build speed of Docker images in CI pipelines. I explain multiple caching-tricks offered by BuildKit, Docker’s integrated image build engine. I also elaborate on how the .dockerignore file and extra arguments to package managers such as apt can speed up your image builds. Originally posted on 2022-01-23, … Read more
This article presents 5 tips that improve the speed of building Docker images locally. It focuses on two problems: iterating on your unfinished Dockerfile (where you still figure out configuration options and which packages to install), and iterating on your code (assuming a finished Dockerfile). I discuss tricks such as using BuildKit’s caching features, the … Read more
This article explores my Kubernetes operator that provisions Azure Pipelines agents as Kubernetes Pods. It comes with an in-depth tutorial for how to use my operator. I also explore all alternative options for running Azure Pipeline agents, and explain why using a Kubernetes-based option beats all other options. Introduction CI/CD pipelines are a cornerstone to … Read more
This article explains BuildKit caching in detail. First you learn background knowledge, e.g. how the local and remote cache differs, and what kinds of cache items BuildKit can store where. Then, BuildKit’s garbage collection is explained in detail. Finally, I present 5 tricks to speed up your image builds, e.g. tuning the garbage collection policies, … Read more
This article explains how Docker builds images with BuildKit, via buildx. It discusses the advantages of BuildKit over Docker’s legacy image builder, and then elaborates on three approaches for calling BuildKit in Docker: 1) using the embedded BuildKit library managed by the Docker daemon, 2) using a BuildKit container managed by buildx, and 3) using … Read more
This article explains how to use “tar” correctly, to backup Docker volumes and restore them. I explain why two top-ranked tutorials are not doing a good job, by taking them apart. Finally, I give hints for creating backups of Docker volumes in production. Introduction Making a local backup of a Docker volume and restoring it … Read more
In this article I present 6 kinds of real-world Docker portability issues, where Docker-based software does not run or build the same way on two different machines. Introduction to Docker portability When I started learning about Docker several years ago, I was very enthusiastic about its promise of portability: write your software and Dockerfile once, … Read more
In this article I present two tools that help you with a Docker image diff or Docker image analysis: “dive” and the slim.ai Docker Desktop extension. I explain use cases for which you need these tools, and I show screen shots that illustrate their usage. Introduction Docker images (or more generally: OCI images) have become … Read more
The Docker build cache avoids rebuilding those parts of a Docker image that were already built. Unfortunately, cache misses are hard to debug. In this article I explain three frequent yet unexpected reasons for cache misses, with solutions. One of them is that COPY or ADD statements are rebuilt, because files have changed. To diagnose … Read more