Docker build cache debug techniques

Docker build cache: debug techniques

The Docker build cache avoids rebuilding those parts of a Docker image that were already built. Unfortunately, cache misses are hard to debug. In this article I explain three frequent yet unexpected reasons for cache misses, with solutions. One of them is that COPY or ADD statements are rebuilt, because files have changed. To diagnose … Read more

infrastructure testing for Ansible roles and playbooks feature

Infrastructure testing for Ansible playbooks and roles: an introductory guide

This article demonstrates how you can do infrastructure testing for Ansible roles and playbooks. I explain how the tools Vagrant and Molecule+Docker let you easily provision temporary VMs or Docker containers in which you can experimentally run your Ansible roles/playbooks, or even run unit testing in Continuous Integration. Introduction to infrastructure testing Ansible is a … Read more

container-based development environments feature

Container-based development environments

In this article I discuss container-based development, where you not only run but also develop and debug software inside a locally-running Docker container. I explain the advantages and disadvantages of this approach, and look at VS Code’s development container feature, various features of IntelliJ-based IDEs, and how to implement an IDE-independent approach. Introduction to container-based … Read more

gitlab vs docker caching

GitLab CI/CD: GitLab vs. Docker caching – how to get insanely fast CI/CD pipelines

Using a Node.js example project I demonstrate how Docker-based caching can speed up your GitLab CI/CD pipelines even more than GitLab’s built-in caching mechanism. I explain how each approach works, and what the technical prerequisites are. I also list tools that support you with setting up a Docker-based CI pipeline. Introduction Caching of files between … Read more

docker image tagging best practices feature

Docker image tagging: best practices in a CI pipeline

This article explains why you should not tag your own Docker images with only the “latest” tag. I discuss alternative best practices, categorizing them into stable vs. unstable version tags. I also fully deconstruct a Docker image tag into its basic components to improve your understanding of Docker image names and tags. Introduction When you … Read more

self-hosted gitlab runner

Operating a self-hosted GitLab runner with Docker

In this article I explain how (and why) you install and use a Linux-based self-hosted GitLab CI/CD runner that executes jobs of your GitLab pipelines. I go into a few caveats and how you can reduce maintenance efforts for the runner to a minimum. Introduction GitLab CI/CD has a distributed architecture that consists of a … Read more

continuous deployment docker

Automatic Continuous Deployment of Docker containers

This article explains how to achieve Continuous Deployment of Docker-based software, using either pull-based approaches (external tools such as watchtower and harbormaster), or push-based techniques (deployment from the CI/CD pipeline). I explain the advantages and disadvantages of each approach, and also illustrate how automated testing greatly reduces the risk of unnoticed failed deployments. Introduction When … Read more

operating docker tips

Tips for operating Docker – the best tools and commands

This article presents useful tips for operating Docker engine on a Linux server. I explain how you can improve the server’s security via automatic updates and scanning the images of your running containers, how to reduce the maintenance efforts by making sure your disks don’t run out of space, and how to set up a … Read more

optimize Docker image size feature

Docker optimization guide: 8 tricks to optimize your Docker image size

This article introduces several tricks that you can apply at build-time, to reduce the size of your Docker images, including the use of a small base image, multi-stage builds, consolidation of RUN statements, avoiding separate chown commands, or using the tool docker-slim. Introduction Docker has become a commodity to package and distribute (backend) software. So … Read more