The complete introduction to Vagrant

Vagrant is a tool that automatically downloads and configures virtual machines, according to a recipe file. It is similar to Docker, but with extended support for different, virtualized operating systems. Use cases include setting up virtual test beds, or CI build machines. This article provides an introduction to Vagrant’s concepts and how it works, as well as many tips and tricks for advanced users.

Introduction

Vagrant is a free and open-source tool for Windows, Linux and macOS that manages virtual machines (VMs) and the software installed into them. It is comparable to Docker, in particular to docker-compose. With Docker, images are typically software bundles that you run on Linux – either natively or inside a VM. In contrast, Vagrant images are (almost) always run in a VM, and several hypervisors (such as Virtualbox, VMWare, Hyper-V, …) are supported. Consequently, Vagrant is truly cross-platform.

You can also think of Vagrant as an automation tool for creating and setting up VMs. All you need to do is write a recipe file (the Vagrantfile, more details below) and distribute it somehow, e.g. via Git, email, etc. The recipient only needs to run one command and a few minutes later, they have a configured VM ready to work with.

In this first article of a three part series, I’ll introduce you to Vagrant and how it works, and provide several insider tips. The second article explains how to create your own Vagrant boxes. A third article explains how Vagrant and Docker are different, and how they can be used together.

Use cases

You might wonder what you need a tool like Vagrant for, so here are a few examples:

  • Evaluating software: no matter if you’re an “end-user” or a professional who wants to test a (time-limited) piece of software: now you can. With Vagrant you set up any virtual operating system, of any version, in minutes.
  • Manual or automated (black-box) system testing: if you make software that runs on several operating systems (and different versions, like Windows 7, 8.1, or 10), maintaining a set of physical machines is expensive and cumbersome. With Vagrant you can quickly create fresh operating systems (in a defined state) and test whether your software successfully installs and runs on those.
  • Continuous Integration (CI): use Vagrant to set up virtual build machines or test runners that contain your required compilers and other tools.
  • Virtual development environments: with Vagrant you can create virtual machines that come with a preconfigured set of tools, allowing your team members to get started with development quickly.

Terminology

Before I’ll go into the details, I’d like to define a few terms from the Vagrant world:

  • Provider: A provider is simply a platform that can create and run VMs. This could be a locally installed hypervisor, e.g. Hyper-V (Windows) or VirtualBox, which are supported by Vagrant out of the box. But Vagrant can also be extended by plug-ins. There are many provider plug-ins, e.g. for Amazon Web Services (AWS) or Microsoft’s Azure, which can also act as providers. Think of them as remote hypervisors.
  • Box: an immutable virtual disk image (and a few meta-data files) of a virtualized system. A box is specifically built for a specific provider. Boxes are stored in a repository. HashiCorp, the maker of Vagrant, runs such a repository, the Vagrant Cloud, which has thousands of ready-to-use boxes for all kinds of operating systems. In this separate article I explain how you can build your own boxes.
  • VM: the mutable copy of an immutable box. Vagrant essentially clones a box and turns it into a VM that you can work in (and whose disk content you can change).
  • Vagrantfile: a regular text file which is the recipe that specifies how Vagrant should build a VM from a box. It references the base box that should be cloned, and contains statements that configure the VM once it has started. This configuration phase is called provisioning in the Vagrant world. A concrete example of a Vagrantfile is shown below.
  • Provisioner: a configuration management software that Vagrant installs into the VM and that facilitates provisioning of the VM. While you could just specify shell commands in the Vagrantfile to install software, these would not be portable, because installing software on different Linuxes, or Windows, works completely differently. Fortunately, Vagrant supports a number of provisioners (and knows how to install their run-times), such as Docker, Chef, Puppet or Ansible.

Vagrant features

Vagrant’s promise is to boost your efficiency when it comes to creating and configuring virtual appliances. The Vagrantfile offers a ton of configuration options, most of which are independent of the concrete provider you are using (such as VirtualBox). This makes it easy to switch from one provider to another, if need be. Here is an excerpt of a few notable configuration options you can place in a Vagrantfile:

  • Choose the box your VM should be based on (e.g. with a line such as config.vm.box = "centos/7"). Vagrant Cloud has a large catalog of boxes. They range from Windows over Linuxes, macOS, etc., and are built for the most common providers like Hyper-V, VirtualBox and VMware Fusion.
  • Define multiple machines in a single Vagrantfile, to set up a cluster of machines.
  • Copy files and folders from the host into the VM, such as installation bundles you want to install, or test data.
  • Execute pre-made shell/batch scripts to provision your VM. Vagrant copies them from the host into the VM temporarily. Alternatively, you can specify shell commands in the Vagrantfile directly.
  • Install software into the VM using a a provisioner such as Chef, Puppet, or Docker. This is much easier than writing OS-specific shell scripts that download and install the software manually.
  • Configure private networks (in addition to the default NAT interface that allows a VM to access the Internet), which allow VMs to connect to one another, via static IP addresses.
  • Configure shared folders between host and VM, which are synchronized in real-time.
  • Configure port forwarding from the host to the VM.
  • Connect into the the VM using SSH or Powershell.
  • Configure provider-specific options: not every provider configuration option is abstracted by Vagrant. For instance, setting the number of CPUs or memory is not abstracted, because these options don’t necessarily make sense for every provider. To solve this, Vagrant allows you to define provider-specific sections where you can set such options.

Vagrant’s functionality does not end here. There is a large number of plug-ins you should check out, once you are familiar with the basics.

First steps with Vagrant

First, I recommend you use Vagrant with a local hypervisor as provider. This means that you’ll have to install one, such as VirtualBox (available on all platforms), or Hyper-V (Windows 8.1 or 10 Pro, see here for installation instructions). To use Vagrant with VMware (Fusion or Workstation), you’ll have to pay, not only for the VMware product itself, but also for the Vagrant VMware plug-in. Thus, VMware might not be your first choice.

Find a base box

After you downloaded and installed Vagrant, you need to determine the globally unique name for the box that you want to instantiate your VM from. To do this, go to the Vagrant Cloud and search for an operating system of your choice. Make sure to filter your search according to your provider / hypervisor. If you go to the details page of a box, you’ll see how to get started. What’s important to understand is that Vagrant creates and boots your VM with the vagrant up command, which expects that it is run in a directory that contains a Vagrantfile. You have two options to create this file:

  1. Manually create an empty Vagrantfile file and paste the content displayed on the box’s detail page, such as
    Vagrant.configure("2") do |config|
    config.vm.box = "generic/ubuntu1904"
    end
  2. Or: run vagrant init generic/ubuntu1904 (or any other box name) in some directory, which creates a pre-initialized Vagrantfile with most features being commented out.

More advanced users would now make changes to the Vagrantfile to set further configuration options, such as the config.vm options documented here. You may want to skip this if you’re still new to Vagrant.

Start the VM

To start the VM, just use the terminal (command prompt on Windows), switch to the directory that contains the Vagrantfile and run vagrant up

Let’s take a look at the magic that happens under the hood:

  1. Vagrant maintains a cache of Vagrant boxes on your machine. If the cache doesn’t contain the box yet (such as generic/ubuntu1904), Vagrant downloads the box to that cache, whose location on the file system is documented here. Boxes can easily have sizes of 5 GB or more, so this will take its time.
  2. Vagrant creates a VM for a specific provider from the box, by cloning the box’s virtual disk image. If you don’t specify a provider, Vagrant assumes VirtualBox by default. You can globally override this with the VAGRANT_DEFAULT_PROVIDER environment variable (e.g. setting it to hyperv), or you can override the provider on a per-project basis with a call such as vagrant up --provider hyperv
  3. Vagrant detects that the VM is not running yet and thus starts the VM using the provider, and configures some default settings. These defaults are provided in another Vagrantfile that box shipped with. Vagrant then loads your Vagrantfile that you created above, which essentially only contains overwrite-options. The vagrant up command has completed once the VM finished booting.

The above steps may vary, depending on the environment. For instance, if Vagrant already has the box cached, step 1 is skipped. Similarly, the VM only needs to be created if you run vagrant up for the very first time, or if you just destroyed the VM. Vagrant will also skip the last step in case it detects that the VM is already running.

Use the VM

How you use the VM depends on the VM’s OS. With vagrant ssh you will get SSH access to Linux or UNIX-based systems, like macOS. A convention in the Vagrant community is that every box should have a user named “vagrant” with password “vagrant”. For Windows-based VMs there is vagrant powershell, vagrant rdp (Remote desktop), and directly accessing the VM graphically. To achieve the latter with VirtualBox, just open the VirtualBox application, find the currently running VM and click the Show button to show the window.

Stop and destroy the VM

To just shut down the VM, use vagrant halt. Vagrant will try to gracefully terminate the VM first, by sending a shutdown-signal into the VM, waiting for the VM to react. If it does not react after some time, it will be forcefully shut down.

To destroy the VM, including its disk image and the entry in the provider (like VirtualBox’s UI), type vagrant destroy. Note that this won’t delete the box from the box cache. To see which boxes are cached, use vagrant box list followed by vagrant box remove xxx to completely remove a box.

Common Vagrantfile configuration options

Whenever I create a new VM with Vagrant, I take the following Vagrantfile template and adapt it to my needs:


# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  # update with actual box name
  config.vm.box = "foo/bar"
  config.vm.network "private_network", ip: "192.168.0.10"
  # src is relative to the Vagrantfile on the host, the second path is the absolute path in the VM
  config.vm.synced_folder "src/", "/var/www/html"

  config.vm.provision "shell", inline: <<-SHELL
    shell commands go here
    can have multiple lines
  SHELL

  config.vm.network "forwarded_port", guest: 1234, host: 1234, host_ip: "127.0.0.1"
  
  # Sets the preferred (not enforced!) provider to be "virtualbox" and configures it.
  # If you instruct Vagrant to use a different provider, this block will be ignored!
  config.vm.provider "virtualbox" do |vb|
    # megabytes
    vb.memory = "2048"
    # to distinguish many VMs in the VBox UI
    vb.name = "My VM Name"
    vb.gui = true
    # number of CPU cores
    vb.cpus = 2
    vb.customize ['modifyvm', :id, '--clipboard', 'bidirectional']
    vb.customize ["modifyvm", :id, "--vram", "128"]
  end
endCode language: Ruby (ruby)

Of course there are many more options you should check out here.

Tips and Tricks

Provider-specific configuration

If you choose Hyper-V as provider, I highly recommend you carefully read this article for further information. If you use VirtualBox, then you should know that there are many more customization options available. An entry like

vb.customize ['modifyvm', :id, '--clipboard', 'bidirectional']Code language: Ruby (ruby)

tells VirtualBox to use the VBoxManage command, which has tons of other parameters – see the official VBox docs. For instance, you could activate traffic shaping (only affecting upload direction) to 1 mbps (as documented here) using these commands:

vb.customize ['bandwidthctl', :id, 'add', 'MyLimit', '--type', 'network', '--limit', '1m']
vb.customize ['modifyvm', :id, '--nicbandwidthgroup1', 'MyLimit']Code language: Ruby (ruby)

However, be careful with these commands, as they may change over time for different VirtualBox major versions (e.g. 5 to 6)!

Configuration options

In the Start the VM section I mentioned that before your Vagrantfile is loaded, Vagrant will load the box‘s Vagrantfile first (official documentation). The box’s Vagrantfile may have several default settings you might not be aware of. To see its content, find the Vagrant Home folder (see here), go into the “boxes” directory, and follow the correct sub-directory corresponding to the box’s name.

Another important detail is that vagrant up will re-apply most configuration settings (e.g. forwarded ports) from your Vagrantfile on every start. However, provisioning statements are an exception: they are only applied on the first VM boot. This page describes which CLI arguments exist to enforce provisioning anyway. You can also override this behavior from within your Vagrantfile by specifying the run: "always" flag. See here for more details.

Synced folders and storage

As documented here, Vagrant makes it easy to set up shared folders. Most of the time, though, you don’t need to configure anything. Most boxes come with a synced folder already set up, which maps the host directory that contains your Vagrantfile to /vagrant or (or C:\vagrant on Windows) in the VM.

Synced folders have various configuration options and alternative drivers, such as NFS, SMB or rsync. They all have different pros and cons, most notably performance, space-consumption (some consume disk space in the host and guest, some only on the host), or availability (e.g. NFS won’t work on Windows hosts). Personally, I never changed any settings, because performance was not an issue for me. If you have such kinds of issues, consult the Vagrant docs for alternatives.

If you use more than one VM based on the same box, use linked clones to reduce disk usage.

CLI usage

Vagrant has a lot of commands. In the spirit of my previous How to boost development productivity article I highly recommend you read through the commands listed on the official docs and create your own cheat sheet. You’ll be delighted once you find out what commands like vagrant suspend or vagrant snapshot can do for you.

Windows evaluation period extension

When using Windows VMs (with Vagrant or otherwise), you need to be aware of licensing and trial periods. Windows licenses are not free. Therefore, Windows boxes you’ll download from Vagrant Cloud are built using evaluation versions. These are time-limited, typically to 90 days, measured from the date the box was built (not: from when you start your “fresh” VM). Once that evaluation period has expired, the VM will still boot, but it will automatically shut down after a few minutes.

You’ll find an indication of how many days are left, by looking at the bottom right corner on the Windows VM’s desktop. Chances are that this number is small (or the license has already expired) right on the first boot, simply because the box may have been built and uploaded several days, months or even years in the past. Fortunately, Windows evaluation periods can be renewed up to 3 times. Each renewal resets the counter to 90 days. To renew the evaluation period, open a command line with administrator privileges, and type slmgr.vbs -rearm which triggers a confirmation dialog. Reboot the machine. It might take 1-2 minutes after the reboot has completed for the counter to update (the one shown on the desktop). You can add the following block to your Vagrantfile near the top, which performs exactly this step:

config.vm.provision "shell" do |s|
  s.privileged = true
  s.inline = 'slmgr.vbs -rearm'
  s.reboot = true
endCode language: Ruby (ruby)

In rare cases, you may even have to reboot the VM a second time (without re-running the rearm command). I’ve also observed that, in some cases (e.g. for Windows 7), the counter did update, but to a very short trial period of 7 days. In this case, activating Windows will do the trick. Simply open Windows’ start menu, type “activate” and select the entry to activate Windows. Once activated, the counter increases from 7 to 90 days.

Use snapshots to iteratively build provisioning commands

Building the list of provisioning commands can be laborious, because once a command is executed (which changes the VM’s state), there is no “undo button” to start over, in case a command was incorrect. Instead of regularly using vagrant destroy + vagrant up to completely restart the VM, it makes more sense to create a snapshot of the VM once it started for the first time (and no provisioning commands were applied yet). Then you just restore snapshots whenever one of the provisioning commands went sideways. The vagrant snapshot command is documented here.

Useful plug-ins and extensions

As I mentioned above, Vagrant can be extended by many different plug-ins. Here are a few of my personal highlights:

  • If you are on Windows and want to use PuTTy as SSH client, then you should install the Vagrant PuTTy plug-in (installation instructions). This will let you connect to your Linux-based box from Windows via vagrant putty in place of vagrant ssh. Without this plug-in, you would have to follow verbose tutorials that explain how to convert the Vagrant-generated SSH keys into a format that PuTTy understands.
  • landrush sets up a DNS server that allows to use the VMs host names (instead of their IPs) both from the host and from the VMs.
  • vagrant-persistent-storage creates persistent storages (virtual disk files) and mounts them into a VirtualBox VM. If you are familiar with Docker, this this is similar to Docker’s volume mounts.
  • vagrant-vbguest installs or updates the VirtualBox guest additions into your VM. This is useful if the base box was built without (or with outdated) guest additions (compared to your installed VirtualBox version).

If you fancy graphical frontends, then take a look at Vagrant Manager, which is Vagrant GUI for macOS. I did not find a similar tool for other OSes, though.

Conclusion

If you work with VMs on a regular basis, Vagrant is a huge time-saver. Downloading virtual disk images is a thing of the past, thanks to the Vagrant Cloud which hosts thousands of boxes. Installing additional software is also done quickly, with Vagrant’s support for various provisioners like Chef or Docker. Vagrant is easy to learn, so pick it up if its functionality sounds useful to you.

Leave a Comment