Unexpected Docker Hub rate limit for HEAD requests

For developers who directly call the Docker Hub APIs, this article sheds light on how Docker Hub handles rate limiting, particularly for HEAD requests. Here, the behavior differs in unexpected ways from the well-known image pull rate limits.

Documented Docker Hub rate limits

It took me a while to understand that there is not just one Docker Hub rate limit, but the existing documentation talks about several unrelated rate limits:

• Limits that apply to requests you make to the (proprietary) Hub APIs, that is, calling endpoints starting with https://hub.docker.com/
• Pull (request) limits (a.k.a. download rate limits) that apply to the Docker Hub registry (following the distribution spec API standard).
  • When you (request to) pull an image from index.docker.io, Docker Hub counts any GET request you make to https://index.docker.io/v2/<image-name>/manifests/<tag>, which retrieves the image manifest, as “image pull request”.
  • Note: To really pull the image, you need to make additional GET requests to https://index.docker.io/v2/<image-name>/blobs/<digest> to retrieve the blobs of the image layers (the compressed archives) and the image config (JSON file).
• Overall Docker Hub registry API limits, documented as follows (as of 12/2024): “Docker Hub also has an overall rate limit to protect the application and infrastructure. This limit applies to all requests to Hub properties including web pages, APIs, and image pulls. The limit is applied per-IP, and while the limit changes over time depending on load and other factors, it’s in the order of thousands of requests per minute. The overall rate limit applies to all users equally regardless of account level.”

In the 2nd case (image pull request limits), when you make GET requests to retrieve the manifest, the total and remaining number of pull requests are indicated via separate HTTP response headers:

• ratelimit-limit: 200;w=21600
• ratelimit-remaining: 193;w=21600

where w=21600 indicates the window size over which the limit is calculated (21600 seconds = 6 hours). The total limit changes depending on whether you are anonymous or logged in. And if you are logged in, the total limit depends on your Docker hub subscription plan (e.g. Personal, Pro, Team).

Docker Hub Rate Limits when making HEAD requests

In any (REST) API, HEAD requests are used to determine whether a resource exists (that is part of the URL). In the case of Docker Hub, making a HEAD request to https://index.docker.io/v2/<image-name>/manifests/<tag> determines whether the Docker image version tag exists (HTTP status code 200 vs. 404). If it exists, there is a HTTP response header indicating the tag’s digest.

This is exactly the information I needed when developing the Docker Tag Monitor (see blog post), which helps you determine the rebuild frequency of Docker images. For Docker Tag Monitor to work as intended, it must be able to make many requests to Docker Hub, without hitting its rate limit.

My experiments have shown that the overall rate limits of Docker Hub (see 3rd point above) apply when making such HEAD requests. As written above, the overall rate limit makes no difference whether you use a Docker Hub account or not. Once you exceed about 2-3 thousand requests per minute, the API returns a HTTP response code 429 (Too Many Requests) with an undocumented “Retry-After: 120” response header, which indicates to wait for 2 minutes (120 seconds).