Create a Vagrant box with Packer

How to create a Vagrant box with Packer

by

Vagrant is a tool that automatically downloads and configures virtual machines, according to a recipe file. Typically, the base image is downloaded from the Vagrant Cloud, a repository of user-submitted boxes. This article explains how to create a Vagrant box, which is necessary if there is no appropriate box on Vagrant Cloud, or if you don’t trust the existing ones. I focus on using Packer, which offers a fully automated build process for boxes, using template files. I will discuss how Packer works, caveats to look out for, and provide guidance for those who want to build Windows and macOS boxes.

Table Of Contents
  1. Introduction
  2. Why build your own boxes
  3. Packer
  4. Tips and caveats
  5. Create a Vagrant box with Packer, using pre-made templates
  6. Conclusion

Introduction

Vagrant is an automation tool for creating and preparing virtual machines (VMs). Please read my introduction article, The complete introduction to Vagrant, for a complete introduction to Vagrant. In this article, which is part 2 of my Vagrant series, I discuss how to build your own Vagrant boxes using Packer, another tool by Hashicorp, the makers of Vagrant. Let’s take a look why we would need that, what Packer is, and how to use it.

Why build your own boxes

One of the main reasons why Vagrant is a great tool for setting up VMs is the Vagrant Cloud. It offers a large catalog of ready-to-use boxes for all kinds of operating systems. However, sometimes you will want to create your own box:

  • The box you need is missing entirely from the Vagrant Cloud.
  • Boxes for the guest OS of your choice do exist on Vagrant Cloud, but not in the right OS version, or not for the right provider (hypervisor).
  • You don’t trust boxes hosted on Vagrant Cloud. The reason is that anyone is able to upload boxes to this service. Hashicorp does not examine the boxes. A bad actor could just embed malware into a box, which tracks your activities in the VM, or sends information to a remote server. Imagine you’re using someone’s box as build machine, and then discover that your application’s source code was leaked to an unknown third party. Ooops…
  • You need a customized box where specific software is already pre-installed. This reduces the provisioning time for a new VM, because that software already comes with the box, thus it does not need to be provisioned.

Packer

Hashicorp’s Packer is a separate tool which create boxes. It performs these steps:

  • Creates a new VM,
  • [Optional] Downloads an installation medium (e.g. an iso image) of the OS,
  • Installs the OS into the new VM, from the installation medium,
  • [Optional] Installs OS updates, downloading them from the Internet,
  • Configures settings in the OS, such as creating a vagrant user,
  • [Optional] Installs custom software packages that you define,
  • [Optional] Optimizes the VM’s file size,
  • Packages the VM as a redistributable box file that you could upload to Vagrant Cloud (if you choose to do so).

These steps run fully automated, and can be repeated for different providers, like Hyper-V or VirtualBox. The recipe that tells Packer how to do these things is called a packer template. It is possible to build such templates yourself, but it requires a lot of skill and expertise. Thus, I recommend you use pre-made templates. The remainder of this article will contain links to a few of those. However, these templates come and go over time. To find well-maintained templates at the time you read this, I suggest you just search the Internet for terms like “packer template <os>”, e.g. “packer template macos”.

Tips and caveats

Before I present some details regarding specific operating systems, there are a few general points to consider before you start using Packer templates:

  • You need to run Packer on a physical machine (not inside a VM) with a lot of free disk space (sometimes 50 GB or more). The installation media, VM(s) and box(es) take considerable amounts of space, at least temporarily.
  • Building a box can easily take several hours, most notably because installing OS updates is time-consuming.
  • Account for a few hours of time to learn how to use Packer in general, and specific aspects of the concrete Packer template you will be using. Some templates offer a lot of (useful) configuration options or helper scripts. If you’re wary of your privacy, read the template’s code to fully build trust, making sure that the template does not install any fishy software into the box.
  • The output of a successful build is typically a .box file. If you want to keep those files on site (i.e., if you don’t want to upload them to Vagrant Cloud, e.g. for performance reasons), you can put them on a network drive (SMB, CIFS) or host them on one some HTTP server. In this case, you have to reference the absolute URL (https://... or file://...) in the config.vm.box_url option in your Vagrantfile. Alternatively, run vagrant box add <name of box> <url> once, to add the box to your machine’s box-cache. Then, use <name of box> in the config.vm.box setting of your Vagrantfile.
  • With Packer you can install additional software into the box, before finalizing it. The spot where you would add your installation scripts (.sh, .bat or .ps1) is located in the template .json file given to Packer, e.g. the windows_10.json for this template. Examine that file’s "provisioners" section, which is documented here. Installing software into a box speeds up booting a Vagrant VM for the first time, because provisioning that software is no longer necessary. However, unless you frequently rebuild your box, the software baked into the box will become outdated quickly. Thus, you may want to limit the use of this option, and consider it only for very time-consuming software installations, e.g. compiler tool chains such as Visual Studio.
  • Beware of version incompatibilities of hypervisors. In particular, some hypervisors are not downward compatible, i.e., if you create a VM (and the resulting box) on a newer version of the hypervisor on host #1, it might not start using an older hypervisor version on host #2. I’ve personally encountered this problem, where a box built with VirtualBox 6.x would not start on a different host that has VirtualBox 5 installed. If in doubt, deliberately use older hypervisor versions on the build machine.

Create a Vagrant box with Packer, using pre-made templates

Creating your own Packer templates from scratch is a lot of work. Hence, I suggest you instead look for templates made by others, and customize them, if necessary. Because each existing Packer template is quite unique, I cannot simply provide a definite guide here. In general, you’ll need to download and install Packer and Vagrant, and install all those hypervisors you want to build boxes for.

Building Linux-based boxes is usually easy, and often even not necessary, in case Vagrant Cloud-hosted boxes are built by the respective official Linux maintainers. For Windows and macOS, however, there are no official images. I will discuss a few helpful pointers for these operating systems next.

Windows

There are various Packer templates for Windows, see e.g. boxcutter, Scherer, gusztavvargadr or idi-ops. Some require that you already have a Windows iso image at hand, others will download it automatically. However, getting evaluation-license images for Windows 10 or Windows Server is not difficult, because they are officially available from Microsoft’s evaluation center.

As for the system requirements when packing: Packer is generally available for every major host OS. However, the template may impose additional restrictions, e.g. because they require the execution of shell scripts. This might still work on Windows if you use WSL, but I did not verify this. If you’re just starting out, I recommend that you try Scherer‘s template. It is well-written and imposes no additional restrictions.

In case you want to install custom software, like Git, web browsers, compilers, etc. on Windows, consider chocolatey. It can be installed with a simple Powershell command, see scripts/chocolatey.bat file in Scherer‘s repository. Once installed, you can install additional software with a simple command, such as choco install visualstudio2019community.

If the Windows box you are building uses a Windows evaluation license (which is typically the default), make sure to read my introduction article, The complete introduction to Vagrant. There I explain how to extend (or rearm) Windows trial periods once they expire, see the Tips and Tricks section.

macOS

Building macOS boxes is a very delicate (fragile) process that only works for a few months, until Apple breaks the process again. All templates require that you run them (and Packer) on macOS hardware, and that you already have the installation medium (as app bundle).

Most Packer templates you find on the Internet are extremely outdated. They are only compatible with specific legacy macOS versions. For instance, the boxcutter macOS template can be used to build boxes for macOS 10.7 – 10.11, and early 10.12 versions. To build more recent macOS guests (e.g. 10.13 – 10.15), the macinbox tool is the most promising approach. It even links to a script (installinstallmacos.py) that will assemble an installation medium, downloading the different parts from Apple directly. Another alternative which is maintained at the time of writing is macos-virtualbox.

To install custom software into the box, I recommend Homebrew for macOS. Like chocolatey, it lets you install a large variety of third party application with a simple shell command.

On a final note: according to macOS’s SLAs (software license agreements), macOS may only be used on “Apple-branded hardware”. That means, that e.g. a macOS 10.15 VirtualBox box that you built may not be started on a Windows or Linux (PC) host. In fact, VirtualBox will actually deny starting the VM. If you like to annoy Apple *cough*, then searching the web for “cpuidset” in combination with “VirtualBox” and “macos” might reveal some insights…

Conclusion

Just like with Vagrant, Packer is all about automation. It lets you create new, well-defined boxes from scratch, from a recipe, the Packer template. Although you will need to invest a few hours into learning Packer and the template’s intricacies, you gain the benefit of full automation. You don’t need to babysit the process, and avoid human error completely.

4 thoughts on “How to create a Vagrant box with Packer”

  2. Am I wrong, or is building a Linux box completely glossed over? It says it’s usually unnecessary, but I can’t find a public arm64 ubuntu box for the vagrant provider, which I need.

    Reply

    • Yes, the combination of ARM and Linux is not something that seems to be covered well by the Internet in general, and I did not need to do it back then when I wrote this article. You’re on your own, unfortunately.

      Reply

Leave a Comment