Renovate Bot: 4 advanced tips and tricks – Part II

This is part two of a series that provides tips for advanced Renovate Bot users. It gives valueable tips for writing your own regex managers, explains why and how to self-host a Renovate Bot instance, how you can reduce friction when introducting Renovate to your teams, and encourages interacting with the active Renovate community.

Introduction to Renovate Bot

Renovate (Bot) is a CLI tool that regularly scans your Git repositories for outdated dependencies. If it finds any, it automatically creates a pull request (PR) that updates the dependency. I highly recommend my Renovate Bot introduction article to get you started with the basics, my cheat sheet article for the first steps regarding your configuration tuning, and of course part 1 of this article series.

This article targets Renovate users who already work with Renovate for a few days or weeks, and would like to know more about some of Renovate’s capabilities. It contains several lessons I learnt over the course of several years of using Renovate.

Tip 1: Write Regular Expression Managers

Sometimes you want Renovate to update dependencies that you defined in some file format that is not supported by any manager shipped with Renovate. The custom regex manager (docs) let’s you find updates in this situation. The official Renovate documentation is very good and includes many examples, so make sure to read it carefully. Still, there are several caveats to be aware of:

• You need a solid understanding of regular expressions. You should already be familiar with concepts like (named) capture groups, or what characters like “.” (dot) mean, or under which circumstances you need to escape them. I highly recommend you copy both the file content (of the file you target with fileMatch) and the matchString into a regex helper application, such as https://regex101.com/, to test and debug your regular expression.
• Beware of additional backslashes you need to add (or remove) for the JSON parser that parses renovate.json. For instance, if you want fileMatch to match files such as “foo.bar“, use something like “^foo\\.bar$” in your renovate.json – the JSON parser “eats” one of the two backslashes. Keep this in mind when copying strings between the regex helper app (where the match string is “^foo\.bar$“) and your renovate.json file.
• When matching files, Renovate is somewhat inconsistent in its use of regular expressions vs. minimatch (glob-like notation): the regex-manager’s fileMatch requires regex, while a packageRule‘s matchFileNames requires minimatch. Some of Renovate’s config options, such as matchRepositories, support both notations.
• If you are wondering why regex managers support both depName and packageName: the difference is explained here.
• Syntax such as "{{#if versioning}}{{{versioning}}}{{else}}semver{{/if}}" (see the examples on the regex manager docs page) will make more sense once you read the templates docs page.
• Regular expressions are very susceptible to noise. For instance, if you are matching a section in a JSON file, the order of matched statements in that file must always be the same order as the one expected by your regular expression. To illustrate this point by an example, suppose you have file with content
  [{"package": "angular", "version": "1.2.3"}, {…}, … ]
  then the regular expression (which expects the package and version fields in exactly that order) would not work if someone from your team added a new JSON object such as
  {"version": "2.1.3", "package": "react"}
  to the list. While that object is still valid JSON, the order of the fields is flipped, and the regular expression won’t match. Make sure to document this brittleness somewhere, to reduce the chance that your team runs into this problem. The Renovate authors are aware of this shortcoming and a JSON-based manager is in the works, see ticket.

Tip 2: Best practices for self-hosting

Even though you can avoid self-hosting Renovate if your code is on github.com, I still recommend self-hosting under any circumstances.

Self-hosting advantages

Self-hosting has the following advantages:

• You can specify your preferred execution frequency (in contrast, Mend’s official Renovate app may visit your repository only once every 12 hours, from personal experience)
• Improved reproducibility: you control which version of Renovate is running
• Improved security: you are not giving a third party (Mend) access to your code
• You can run post upgrade tasks (discussed in part 1 of this series)

In practice, I found only one real disadvantage of self-hosting Renovate, which actually only applies if Renovate scans many repositories: you need to build your own approach for scaling the execution of Renovate, once it takes too long to process all repositories (see this discussion for some pointers). Also, if you do not want different teams to know about another, you must build custom tooling that allows teams to access Renovate’s debug logs, where the log output of other teams is redacted. Giving teams access to the debug log outputs is a good idea in general, because the team can then inspect the logs themselves first (before coming to you, the “Renovate expert”), if they suspect something is wrong.

I generally recommend you use scheduled CI pipelines pipelines to execute Renovate (over other approaches, such as cron jobs), and that you run Renovate as Docker/OCI container. Advantages are:

• You most likely use some CI/CD platform anyway, so there is little extra effort to set up a scheduled pipeline
• You get notifications in case of problems with Renovate (e.g. emails sent when a pipeline fails)
• You can easily inspect Renovate’s output log by clicking on executed pipeline instances in the CI/CD platform’s web UI
• You get a high level of reproducibilty, due to pinning the Docker image version

If you self-host on GitLab, you can use Renovate’s official GitLab runner repository. In all other cases, you can either:

1. Run a container-based job (which most CI/CD platforms support), or
2. Run docker build … followed by docker run --rm … to locally build (but not push) and then run the image

There are two official Renovate Docker image variants: the full image and the slim image, see the docs for details. Both Docker images are based on the containerbase project and let you install additional tools via the shell command “install-tool <tool-name> <tool-version>“, e.g. “install-tool node 20.0.0“. If you want to know how it works under the hood, the source of the install-tool shell script is available here. Please note that install-tool supports only specific tools, see here for a list. The slim Docker image only comes with a few tools installed, see here, and is therefore smaller. The full image contains many more (but not all) tools, see here. Both these images are based on this multi-stage Dockerfile that defines which concrete tools come pre-installed. Running the install-tool to install additional tools (before running the renovate CLI) may be useful in certain situations, e.g. if you run post upgrade tasks that require such tools (like Helm or Terraform).

Tip 3: Preview Renovate’s PRs on a forked repository

If you introduce Renovate to your software development team which may never have worked with such automated dependency tools before, it is a good idea to start with a copy of the affected repositories. The copy is your safe playground in which to build and tune the renovate.json configuration file, without the risk of generating the wrong PRs. The dev team also gets a full preview of how Renovate’s PRs look, compared to the short list presented by Renovate’s Onboarding PR. Once you have a team agreement to use Renovate, you just delete the repository copies and configure Renovate to visit the real repositories.

To actually make a repository copy, you can either fork it (in this case make sure to read Renovate’s fork processing docs), or you create a new Git repository, set it as an additional Git remote in your local clone of the source repository, and push the default branch of the source repository to the new repository’s remote.

Tip 4: Get help from the very active community

If you are stuck on a problem, do not hesitate to ask questions in Renovate’s official discussion board on GitHub. Chances are that you get responses within minutes or hours, and very often from the maintainers themselves. I have never seen any other open source project with such a high attention to their community. 

Naturally, I recommend you use the search functionality first, before asking a new question.

Conclusion

I hope the tips presented in this article help you get more out of Renovate. Especially tip #2 should help you with setting up your own Renovate instance, which has many advantages. You can find more tips and tricks in part 1 of this series.