Build runners, which compile and package your software, are often implemented using Docker containers. However, you cannot use Docker if you need a Windows or macOS build runner. This article presents how to create a build runner VM with Vagrant, for Windows and macOS. While the instantiation of a build VM is fully automated, it is challenging to create the list of provisioning commands that install and configure your build tools within the VM. This article is part 2 of 2, presenting several tips and recipes for common provisioning steps. Make sure to read part 1 first!
Tips and recipes for how to create a build runner VM with Vagrant
Download and extract archive files
It’s not always possible to install everything with package managers. Sometimes you need to download a zip archive and extract it.
On macOS, this is very easy, because the corresponding tools come with the system. A snippet as the one below will do the job for you:
config.vm.provision "shell", privileged: false, inline: <<-SHELL
curl https://some-url.com/some-archive.zip -L -o local-path.zip
unzip local-path.zip
SHELL
Code language: Ruby (ruby)
The -L
argument ensures that curl
follows redirects. In case the archive is a tar.gz
archive, you will need to usetar xvzf archive-path.tar.gz
instead of unzip
.
On Windows there are no built-in tools (exe binaries) which download or extract archives. However, the Powershell offers so-called cmdlets which do the job for you. Vagrant uses the Powershell anyway for any instructions placed inside an inline
shell block. The following snippet illustrates how to download and extract zip archives:
config.vm.provision "shell", privileged: true, inline: <<-SHELL
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest https://some-url.com/some-archive.zip -OutFile local-path.zip
Expand-Archive local-path.zip -DestinationPath "C:\\libs"
SHELL
Code language: Ruby (ruby)
The first line is only required once per shell
-provisioning block. It enables downloading files via HTTPS, which would otherwise fail if that line was missing. The second and third line download and extract the file respectively. Note that backslashes only need to be escaped because we are using the inline
mechanism of Vagrant. If you instead placed the content inside a .ps1
file, single backslashes would suffice.
Note that the Expand-Archive
cmdlet may not work on older Windows versions, e.g. Windows 8.1. If you run into problems with that command, you need to upgrade Powershell to the latest version. You can use Chocolatey for this, as the following provisioning block illustrates. Place it above the block that calls Expand-Archive
, so that Powershell is updated before using the cmdlet:
config.vm.provision "shell", reboot: true, inline: 'choco install powershell -y --no-progress'
Code language: JavaScript (javascript)
The reboot: true
part is important, as the Powershell upgrade requires a reboot. Also, don’t forget to also install Chocolatey beforehand!
Build and use C++ libraries with CMake
If you use C++ in your project, chances are that you are using CMake to build your project, and to #include
and link other native libraries used as dependencies in your project. Consequently, your build VM needs to download, build and install all those dependencies. While there are projects such as vcpkg which help you do that, they may lack support for pinning specific versions of the libraries, and always install the latest version. The following snippet illustrates the manual installation process for the ZeroMQ library, pinning the version to v4.3.3. For Windows:
config.vm.provision "shell", privileged: true, inline: <<-SHELL
mkdir C:\\libs
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest https://github.com/zeromq/libzmq/releases/download/v4.3.3/zeromq-4.3.3.zip -OutFile C:\\libs\\zeromq.zip
Expand-Archive C:\\libs\\zeromq.zip -DestinationPath "C:\\libs"
cd C:\\libs\\zeromq*
cmake -G "Visual Studio 16 2019" -A x64 -B build64 -D ZMQ_BUILD_TESTS=OFF
cmake --build build64 --target install --config Release
SHELL
Code language: Ruby (ruby)
Similarly, for macOS:
config.vm.provision "shell", privileged: true, inline: <<-SHELL
mkdir libs
cd libs
curl https://github.com/zeromq/libzmq/releases/download/v4.3.3/zeromq-4.3.3.zip -L -o zeromq.zip
unzip zeromq.zip
cd zeromq*
cmake -G "Unix Makefiles" -B thebuild -D ZMQ_BUILD_TESTS=OFF
cmake --build thebuild --target install --config Release
SHELL
Code language: JavaScript (javascript)
A few notes:
- On Windows, the
"Visual Studio 16 2019"
generator will find the Visual Studio Build Tools I discussed in part 1. You don’t need to install the full version of Visual Studio. On macOS, the Command Line Tools will suffice in order to use the"Unix Makefiles"
generator. You only need to install Xcode if you want to use the"Xcode"
generator. - On Windows, you have to decide whether you want to build for 32 for 64-bit. In CMake, 32-bit builds are the default, unless you specify the
-A x64
argument. If you need to support both 32 and 64-bit, things become more complicated. You need to run the last two commands twice (one for each bitness), but take extra care that the secondinstall
command does not overwrite the binary files of your library. I have observed the weird behavior of CMake to always install the headers and binaries somewhere below “C:\Program Files (x86)”, even for 64-bit builds. Check theCMakeCache.txt
file inside the generated build directory of your 64-bit build to verify that theCMAKE_INSTALL_PREFIX
is correctly pointing to “C:/Program Files”. If it is not, delete the build directory and re-run thecmake -G
command, appending something like-D CMAKE_INSTALL_PREFIX="C:/Program Files"
.- Note: on macOS your OS and hardware and underlying build system decides the default architecture (e.g. 64-bit) that is used to during compilation. See here how to build universal binaries, which becomes necessary once you want to natively support Apple silicon!
- The
-B
argument specifies the path to the build directory that CMake will create for you. I simply specified a simple name, i.e., a relative path, which is not used on the file system yet. I recommend against using “build”, as some packages already have a non-empty “build” directory. - The last line builds and installs the ZeroMQ library. The path given by the
--build
argument must match the path provided to the-B
argument from thecmake -G
line.
Silent Git clones and pulls
If you use Git as your Version Control System, your build VM needs to be able to clone
or update (pull
) your code without user interaction. For this, you need to deposit the credentials in a file and put it in the VM, and configure Git to use that file. The following snippet illustrates the necessary additions to your Vagrantfile
:
config.vm.provision "file", source: ".git-credentials", destination: "/Users/vagrant/.git-credentials"
config.vm.provision "shell", privileged: false, inline: <<-SHELL
git config --global credential.interactive never
git config --global credential.useHttpPath true
git config --global credential.helper store
git clone https://yourhost.com/your-project.git
SHELL
Code language: Ruby (ruby)
So, what is going on here?:
- We configure Git to use the
store
credentials-helper, which expects that there is a.git-credentials
file in the user’s home directory. This is a simple text file which contains the usernames, passwords and corresponding hosts / URLs, in clear-text (unencrypted)! You must not store this file in your VCS. For extra security, I recommend that you generate extra Git accounts (with read-only permissions) specifically for your build VM, e.g. using deploy tokens which systems such as GitHub or GitLab offer. This way, even if these credentials get leaked, the damage is limited, as long as you are using a unique, generated password for that Git account.- This
.git-credentials
file is placed in the VM using thefile
provisioner in line 1. Make sure to adapt thedestination
path if you are creating a Windows build VM. - The required format of the content in that file depends on the
credential.useHttpPath
setting (which isfalse
by default). Ifcredential.useHttpPath
is set totrue
, it must contain lines of the formhttps://user:pw@yourhost.com/your-project.git
, otherwise the path postfix is omitted, so each entry should have the formhttps://user:pw@your-host.com
- This
- The
credential.useHttpPath
setting is only needed in case you are using several distinct credentials for several repositories which are located on the same host, e.g. when using Git submodules. Otherwise you can omit that line. - The
credential.interactive
setting ensures that Git really never ever asks for credentials interactively via a GUI or CLI prompt. Especially on Windows I found that, without this setting, Git would still attempt to ask you for the credentials, even though thestore
helper is configured and the credentials are found in the.git-credentials
file. - The
privileged: false
part is important! It avoids that thegit clone
command runs as root user, for which no.git-credentials
file was prepared!
Working with secrets
By “secrets” I mean things like credentials (typically username+password combinations), license keys, or any other data that requires protection. Sometimes, CLI provisioning commands require these kinds of secrets. However, you cannot put these secrets into your Vagrantfile
, because the Vagrantfile
is part of your VCS, and secrets don’t belong in VCS, ever!
To solve this problem, I use simple text files, which contain the secrets. Let’s take a look at an example for macOS, where my build script notarizes my macOS app automatically. This requires my Apple developer account email address and a token, which I don’t want to permanently store on disk, or hard-code into the build script. Instead, I use macOS’s keychain. My provisioning script needs to add an account to the login key chain. I store the secrets, my email address with the secret token, in a text file, e.g. named notarization-credentials
. I add notarization-credentials
to my .gitignore
file, to ensure that it won’t be accidentally committed, and I commit a file such as notarization-credentials.example
instead. It contains dummy data (to illustrate the format), together with instructions on how to get this data, or who to ask, e.g.:
user.name@yourcompany.com;yourPassword
Copy this file to "notarization-credentials".
Make sure the file only contains one line with your own credentials. Create the password as described on
https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution/customizing_the_notarization_workflow
using an "app-specific password". See also https://support.apple.com/en-us/HT204397
Code language: JavaScript (javascript)
Here is an example for the Bash script (committed to Git) that Vagrant executes to create the account, which reads the credentials from notarization-credentials
:
#!/bin/bash
security unlock-keychain -p vagrant /Users/vagrant/Library/Keychains/login.keychain
EMAIL=$(awk -F ';' '{print $1 }' notarization-credentials)
PASSWORD=$(awk -F ';' '{print $2 }' notarization-credentials)
security add-generic-password -a $EMAIL -s AC_PASSWORD -T /Applications/Xcode.app/Contents/Developer/usr/bin/altool -p "$PASSWORD"
Code language: JavaScript (javascript)
This example uses the command awk -F ';' '{print $1 }' <file>
which reads and parses <file>
, splits the content by the separator specified by the -F
argument, and then outputs the first one ($1
). The EMAIL=$(...)
construct ensures that the output is stored in the variable EMAIL
, which is then accessed in the last line, using $EMAIL
. If all of this is very confusing, you will need to become familiar with basic (Bash) shell programming first.
If your file only contains a single piece of information, you can simply use $(cat <file>)
instead of awk
.
After the secrets have been processed by your shell provisioning scripts, make sure to delete the corresponding files from inside the VM, using another shell provisioning block!
Conclusion
Whenever you need build systems or test runners, you should rely on automation to get reliable, reproducible build or test environments. If you need to work with platforms not supported by Docker, Vagrant offers a great, VM-based alternative. This article, together with part 1, presented a whole slew of pitfalls and tips. I hope that it saves you from having to research related issues.
I find Vagrant to be the most flexible tool for creating build and test environments. The provider (“hypervisor”) is configurable completely independently from your provisioning commands. Thus, the VM could be local, e.g. using VirtualBox, or in the cloud. If you don’t like package managers like Chocolatey or Homebrew, you can also use configuration management tools such as Puppet, Chef or Ansible, which are all integrated with Vagrant. The choice is yours!