This article takes a detailed look at Cosign, one of several tools for creating and verifying Docker images and adding attestations such as build provenance.
Docker Image signing and attestation series
This article is part of a multi-part series:
- Overview: explains concepts and tool landscape, provides high-level recommendations
- Docker Image signing with Notation: detailed analysis of the Notation tool and its signatures
- (This article) Docker Image signing and attestation with Cosign: detailed analysis of Cosign, for image signatures and attestations
- Docker Image attestation with GitHub attestations: detailed analysis of GitHub attestations and its pros&cons
- Docker Image attestations with BuildKit: detailed analysis of BuildKit’s attestations
Introduction
Cosign is one of several open-source tools developed in the Sigstore project. Sigstore is a graduated OpenSSF-backed project with the mission to provide free and effortless build provenance to as many artifact registries as possible. Thus, it is not limited to Docker/container images. For instance, Python packages can now be pushed to PyPi with Sigstore signatures (see announcement), and the macOS package manager Homebrew is also adopting it (announcement).
Under Sigstore, various open source projects (other than Cosign) have been created, e.g. Fulcio & Rekor (that simplify “keyless signing”), Policy Controller (to verify signed images in Kubernetes), gitsign (to use keyless Sigstore signatures to sign Git commits), or Sigstore SDKs for various languages (so that you can build tools that sign or verify arbitrary artifacts).
Certificate-based vs. keyless signing
Certificate-based signing approaches (used by tools like Notation or Docker Content Trust) have various problems that Sigstore aims to solve:
- Identity: as a consumer, you want to verify that the real (expected) publisher signed an artifact. Doing this via (long-lived) public-private key pairs is just a proxy, where possession of the private key is (supposedly) proving your identity. This proxy metaphor breaks down once a malicious actor (e.g. a hacker) stole a copy of the private key, and can henceforth claim to have the identity of the real publisher.
- Key management: keeping private keys secure (so they cannot be lost or stolen) is difficult. Similarly, it’s difficult to make the public key easily accessible for users, while also protecting it from being manipulated by a malicious actor.
- Key revocation: If the key pair is compromised, how do you distribute new keys in a way that convinces users of your legitimacy and that you’re not an attacker?
To solve these problems, Sigstore introduces “keyless” signing, which no longer needs long-lived key pair/signing certificates.
This is a simplified summary of the keyless signing flow:
- When you run “
cosign sign <artifact>”, Cosign generates a new public-private key pair, keeping it only in memory (not storing it on disk). - Cosign initiates an OIDC flow to get a (cryptographically verifiable) OIDC token (that attests the identity of whoever runs
cosign) from one of the many OIDC issuers that Fulcio already trusts, see here for the complete list. This flow could either be completed by a human who manually logs into their Microsoft/GitHub/ etc. account, so the OIDC token would include the human’s email address and the URL of the chosen OIDC token issuer. Or it could be a service account token using fully automated means (without human interaction), e.g. a GitHub Actions runner could get such a token from GitHub’s own OIDC issuer (https://token.actions.githubusercontent.com), where the identity of the OIDC token is the full URL to the GitHub Actions workflow that runs “cosign sign”. - Cosign sends a Certificate Signing Request (that contains the OIDC token and the in-memory public key) to a Certificate Authority, Fulcio. Fulcio validates the OIDC token and returns a temporary signing certificate that is only valid for 10 minutes. Fulcio adds that certificate to its public immutable Certificate Transparency log. By default, Cosign uses a free-to-use “public good” instance of Fulcio, but you could also self-host Fulcio on your own servers, if necessary.
- Cosign signs the artifact using the in-memory private key, and uploads the artifact’s digest, signature, and signing certificate to another public log, the Rekor ledger (again, using a public good instance by default, or you could self-host it).
As a consumer, when you run “cosign verify <artifact> --certificate-identity <email-address-or-URI> --certificate-oidc-issuer https://…”, Cosign performs these steps:
- Retrieve the signature from the image registry
- Verify that the signature is cryptographically valid
- Check with Rekor that the signature is known there. If it is, also check with Fulcio that the certificate is known to Fulcio.
- Verify that the identity (that is part of the signature’s certificate) matches the provided CLI arguments
(--certificate-oidc-issuer,--certificate-identity)
In summary, these are the pros and cons of keyless signing:
- Cons of keyless signing: single point of failure of signing infrastructure (Fulcio and Rekor). If that infrastructure is down, you cannot verify signatures. Self-hosting options are available but suffer from the same problem.
- Pros of keyless signing: simplified key management and verification – the consumer only needs to configure two (human-readable) strings: the issuer URL and the identity (email address or URL)
Deep dive
Let’s examine the commands and the manifests or ledger entries they produce.
Signing images
To sign an image (using the keyless approach), run a command such as
“cosign sign [--recursive] registry.com/some/image@sha256:1234…”
where the hash points to either an image manifest (i.e., an image for a specific OS+CPU architecture) or an image index (which was created for multi-platform images, in this case, you should use cosign’s --recursive flag to instruct cosign to recursively sign the platform-specific image manifests referenced by your provided image index).
As discussed above, if you run the command in an interactive terminal, cosign first asks you to log in with a supported SSO provider (e.g. Microsoft) in the browser, to retrieve your OIDC token. Or, if you run cosign in a CI/CD pipeline, it will auto-detect available issuers (such as GitHub or GitLab) and retrieve the OIDC token without user interaction.
Cosign then creates various manifests and pushes them, creating a new tag named “sha256-<1234…>.sig”, as described in the Cosign Signature Specification. Cosign does not modify the manifest(s) of the referenced image it signs.
Verifying images
To (manually) verify a signed image that was signed using keyless signing, use a command such as “cosign verify <registry.com/image:tag-or-hash> --certificate-identity some@mail.com --certificate-oidc-issuer https://foobar.com”.
The verification only makes sense if you, the verifier, know who was supposed to sign the image. If you used incorrect values for --certificate-identity or --certificate-oidc-issuer, cosign will print an error message, which includes the identity (subject) and issuer of available signatures. If these values look correct to you, repeat the cosign verify command with the issuer and identity values from the error message.
Key-based signing & verification
Cosign also supports signing and verifying images using the “traditional” key-based approach, using a self-generated, long-living public-private key pair. This avoids the need for Rekor and Fulcio and can be appropriate for network-isolated environments, e.g. in an enterprise/corporate setting. This “offline signing” works as follows:
- “
cosign generate-key-pair” creates the files cosign.key and cosign.pub locally. Alternatively, you can also generate the key pair in a cloud-based KMS, e.g. Azure KeyVault, see the documentation for details - “
cosign sign --key cosign.key <image-reference> --tlog-upload=false” signs the image, without making entries in Rekors transparency log - “
cosign verify --key cosign.pub <image-reference> --insecure-ignore-tlog=true” verifies the image
Creating attestations
Using the command “cosign attest --type <predicate-type> --predicate <path-to-attestation-file> <image-ref-with-tag-or-hash>” you tell cosign to package the content of the attestation file (e.g. a CycloneDX-formatted SBOM, see --predicate argument) in some wrapper manifests. Cosign also signs them cryptographically, and uploads them to the image registry, creating a new tag named “sha256-<image-hash>.att” for your image (notice the “.att” ending, in contrast to the “.sig” ending that Cosign creates when signing an image, as explained above). Cosign leaves the manifests of the referenced image itself as-is. This is the same behavior cosign has when signing images.
The file format that Cosign uses for the (wrapper) manifests is in-toto attestations. The “in-toto lingo” uses the term “Predicate” for the claim you want to attest (e.g. the SBOM). In-toto attestations can be generic, but the in-toto attestation framework also predefines various Predicate types, see here for a list. Cosign maps the parameter --type to these predefined Predicate types (--type can have the values slsaprovenance, slsaprovenance02, slsaprovenance1, link, spdx, spdxjson, cyclonedx, vuln, or openvex). You can also set --type to an arbitrary string, typically a URI such as “https://example.com/CodeReview/v1“.
Let’s take a look at the data created by the command “cosign attest --predicate some.json --type spdxjson ttl.sh/aalptest@sha256:c10f729849a3b03cbf222e2220245dd44c39a06d444aa32cc30a35c4c1aba59d”:
As you can see, the structure of the “.att” manifest is the same as the one of a signed image (the “.sig” manifest).
Generic attestations
You can also omit the “--type <attestation-type>” argument. Then, Cosign sets the predicateType to “https://cosign.sigstore.dev/attestation/v1”. I would only do so if you want to attest something that is not (yet) part of the predefined in-toto predicate types. The file can even be a binary file. In contrast, if you specify a Predicate --type Cosign will parse the predicate file and validate that it is valid JSON.
Verifying attestations
To verify the authenticity of the signature of an attestation, run the command “cosign verify-attestation <image-ref-with-digest> --certificate-identity some@mail.com --certificate-oidc-issuer https://…”
It’s also possible to validate the content (and Predicate type) using the CUE or Rego language. See here for examples. Alternatively, you can use the command
“cosign download attestation <image-ref-with-digest> [--predicate-type] > output.json”
to download the attestation (which has the same data format as the manifest of 0635641a52…).
Automated verification
Verification of image signatures (and attestations) created by Cosign/Sigstore is supported by the following open-source projects, which are all Kubernetes operators. Their default behavior is an “enforce mode” which ensures that the images are not deployed to your Kubernetes cluster in case their verification fails. Alternatively, these operators have an optional “audit” mode which only warns you about failed validations, but will still deploy the (invalid) images.
- Kyverno (docs): supports verifying Cosign-created image signatures and attestations
- FluxCD can verify Cosign-created image signatures (docs)
- OPA Gatekeeper with Ratify (see here): supports verifying Cosign image signatures
- Note: while Ratify supports validating SBOMs and vulnerability reports, Ratify requires that you attach these attestations via “
oras attach” (which creates a tag of the form “sha256-<digest-hash>”). Since Cosign uses a different form of attaching the attestation (where the tag is named “sha256-<digest-hash>.att”), Ratify does not recognize the attestations and thus cannot verify them.
- Note: while Ratify supports validating SBOMs and vulnerability reports, Ratify requires that you attach these attestations via “
Each solution uses a slightly different syntax or approach to configure the “trust policy”, which defines which signing certificates/identities you trust.
Another solution is configuring Harbor (see docs) to refuse image pulls of unsigned images. Unfortunately, you cannot configure a trust policy in Harbor, meaning you cannot configure the specific signing identities you deem trustworthy. This makes the feature rather useless.
Finally, you can run “cosign verify” in CI/CD workflows. However, there is the “hen and egg” problem that you need to obtain a genuine cosign binary to begin with before you can use it. This could be as easy as hard-coding a curl download and hardcoding the binary’s corresponding checksum and verifying it. However, this would require regular maintenance. You can instead use a more elaborate “bootstrap approach”, as done by Cosign’s installer GitHub action, see here for details.
Conclusion
In the area of signing container images or adding attestations, Cosign is a very solid and easy-to-use solution. It has widespread adoption by other projects (that verify Cosign signatures), e.g. by Kubernetes Operators like Kyverno.
I should mention that you can also use Cosign to sign ordinary files (“blobs”), see the documentation.
Impending change of signature format
As this write-up explains, “cosign sign”, so far, uses a Cosign Signature Specification that describes how image signatures are represented and implemented (with media-type
“application/vnd.dev.cosign.simplesigning.v1+json”, as seen above).
Since then, Sigstore created a more elaborate Cosign Bundle Specification for attestations.
As of 03/2025, it seems that:
- “
cosign sign” will sooner or later use that attestation bundle spec to encode image signatures. This means that you will need to resign your images once that happens - if you sign blobs (using “
cosign sign-blob” that signs local files), you can use the new bundle format already today, using the CLI argument “--new-bundle-format=true“