Every Release Build Needs an SBOM for the LLM Gateway

An LLM gateway is often treated as a thin HTTP service: it accepts a request, chooses a model, and passes back the response. In production, it is one of the components with the most dependencies. It accepts external traffic, stores provider keys, parses JSON, writes audit logs, applies PII masking, and works with queues, databases, SDKs, and the container platform. A flaw anywhere in this chain can affect every model call.

That is why an SBOM for an LLM gateway is not just an audit folder. It should answer an uncomfortable question within minutes: «Which exact build is processing requests right now, what is it made of, where else is it running, and how do we stop it?» If the team has to open several repositories, search for a registry tag, and debate which image reached the cluster, its SBOM exists only on paper.

An SBOM records the release contents, not the team's intentions

An SBOM should describe what you installed, not what the developers intended to install. It needs a component identifier, version, source, hash or another integrity check, license, relationships with other components, and information about the release itself.

SPDX defines an SBOM as a set of elements describing a package and supports sharing its contents, provenance, licenses, and information about quality or security issues. In the specification, relationships between elements are part of the model, not a decorative field. This matters for a gateway: a library may have entered the image directly, through a provider SDK, or through a system package in the base layer.

Many teams take a simpler approach: they generate JSON from the source directory, attach it to the release, and consider the task complete. Such a file does not know which dependencies a multi-stage build excluded, which system packages came from the base image, which binary an installation script added, or what actually ended up in the final layer. It is useful as an early signal for developers, but it is not suitable for deciding whether to revoke a running image.

Separate the four things that are often called «the contents»:

• a lockfile fixes dependency resolution in a language ecosystem;
• a source SBOM describes the dependency tree before the build;
• a container SBOM describes the contents of the shipped image;
• provenance connects the image to the source revision, environment, and build process.

If these artifacts are confused, the team gets a false sense of confidence. For example, requirements.txt may be tidy while the final image contains an outdated openssl from the base distribution. Or the repository scanner may see a library in a test dependency and block a release even though it never enters the production image.

An LLM gateway contains more than the lockfile

For an LLM gateway, the unit of control should be the executable request path, not one repository. The inventory should include components involved in accepting, processing, routing, and logging requests.

Start with the service's final image. It usually contains the language runtime, HTTP server, TLS libraries, JSON parser, client SDKs, tokenization libraries, storage drivers, telemetry agent, and system packages. If the gateway supports local models, expand the list to include the inference server, CUDA or other acceleration libraries, weight loaders, and the model artifacts themselves.

Then add adjacent artifacts, but do not merge them into one shapeless document. Track these separately:

• the gateway image;
• the worker image, if it processes asynchronous jobs;
• the proxy or adapter image, if your team releases it;
• an extension package loaded at startup;
• the weights file and configuration for a local model.

A model weights file is not a Python library, and a tokenizer is not a container. They need similar disciplines: an immutable identifier, a cryptographic hash, a source, a license, an approval procedure, and an owner. But a separate model inventory is more useful than trying to hide a gigabyte-sized artifact in a regular package SBOM. Otherwise, when someone asks «Which version of the weights was running on Monday?», they will see a long list of apk and pip packages instead of an answer.

Do not forget configuration that changes the attack surface. SBOMs usually do not record secrets, complete URLs, or the contents of rules. But a release should refer to a configuration schema version and the digest of a trusted configuration package. This makes it possible to distinguish the same image running with TLS verification disabled from the same image running with the correct configuration. Secrets and personal data must never enter an SBOM.

Versions must be pinned so the build can be reproduced

An image tag, a floating library version, and a script downloaded from the network make a release irreproducible. A build that used one set of bytes yesterday may produce different bytes tomorrow under the same name. For a gateway with provider keys and external traffic, that is a bad trade.

Pin dependencies at three levels. At the language level, use a lockfile with artifact hashes where the ecosystem supports them. At the container level, specify the base image by digest. In CI, pin the versions of the SBOM generator, scanner, build image, and actions started by the pipeline.

A familiar-looking bad Dockerfile:

FROM python:3.12-slim
RUN pip install openai fastapi uvicorn
COPY . /app

It does not tell you which exact Python version or which wheels entered the image. Tags change, and pip install resolves dependencies during the build. Even if the application behaves the same way, investigating an incident becomes an attempt to reconstruct the past from external repository logs.

The basic principle for a production build looks different:

FROM registry.example/base/python@sha256:<pinned-digest>
WORKDIR /app
COPY requirements.lock ./
RUN pip install --require-hashes -r requirements.lock
COPY . ./

The placeholder is not a ready-to-use configuration. The team must replace it with the actual digest from a trusted internal registry and update it through a controlled pull request. --require-hashes is useful because it stops the installation when a downloaded package does not match the approved contents. It does not replace vulnerability scanning, but it makes it harder to quietly substitute an artifact under a familiar version.

The popular advice to «update everything every night» sounds safe, but it reduces control. An automatic pull request for an update is reasonable. Automatically releasing a new dependency set to production without evaluating the changes creates another risk: you do not know which change caused a routing error, increased latency, or an SDK incompatibility. An update should leave a trail: the source revision, old and new digests, the SBOM difference, test results, and the release decision.

One SBOM per repository cannot answer questions about the image

Generate the SBOM after building the final image and bind it to an immutable digest. This is the file an on-call engineer needs for a CVE, an auditor needs during a review, and an admission controller needs before a cluster launch.

Syft can build SBOMs for images and filesystems and output SPDX and CycloneDX. In practice, I would start with one format that your scanners and storage system can process, and add a second only when a consumer requires it. The tool itself demonstrates this scenario: it creates SPDX JSON and CycloneDX JSON from one image.

An example command for a local check:

IMAGE=registry.example/llm-router@sha256:<digest>
syft "$IMAGE" -o spdx-json=sbom.spdx.json
sha256sum sbom.spdx.json

The expected form of the second command's output is:

8f1c...c92a  sbom.spdx.json

Store this hash together with the image digest. The JSON itself cannot be considered immutable just because it sits in CI artifacts: many systems clean those artifacts up after a retention period, and some allow a file with the same name to be replaced. Storage should retain release artifacts for as long as you are required to support or investigate the release.

In CI, add a check that the SBOM was generated from the digest, not from a tag. Suppose the pipeline built registry.example/llm-router:build-482. It first publishes the image, then obtains its content digest from the registry, and only then generates the SBOM for the @sha256:... reference. If the generator reads a tag, a parallel push can change the contents before scanning. This race is uncommon until the first incident and very unpleasant afterward.

You can release a second, auxiliary SBOM for the source. It helps find risks before the container build and compare changes in a pull request. Name it honestly, for example source-sbom, and do not use it in place of the release image SBOM.

A scanner finds known issues, while policy makes the decision

A scanner matches components against a database of known vulnerabilities. It does not understand your data flow, know whether the code is reachable over the network, or decide by itself whether a release should be disabled. That is why the rule «any CVE blocks everything» quickly becomes a queue of false alarms, while «we will deal with it later» leaves dangerous builds running for months.

NIST's SSDF recommends incorporating secure practices into the development lifecycle to reduce vulnerabilities in released software, limit the impact of unresolved issues, and address the causes of recurrence. This is not an instruction to install one scanner. It is a requirement for a process in which every scan result has an owner and an action.

An LLM gateway policy should account for at least four properties of a finding:

• whether a fixed version exists and can be used without changing the API;
• whether the component is present in the final image;
• whether the vulnerable code is reachable through external requests, an admin interface, or file processing;
• whether the issue affects secrets, tenant isolation, auditing, or code execution.

Consider a typical failure. A scanner finds a vulnerability in an archive library inside the base image. The team adds an exception because «the gateway does not accept archives». Two weeks later, the team adds an endpoint for uploading batch jobs, the library starts processing incoming ZIP files, and the exception remains. What is needed is not a permanent ignore flag, but a record with a reason, owner, review date, and cancellation condition, such as «the exception is valid only while the production endpoint does not accept archives».

The opposite extreme is dangerous too. If CI fails on a vulnerability in an unused package and nobody can ship fixes, engineers will start disabling the scanner entirely. Set separate thresholds: block a release for a critical or high-severity issue in a reachable production component, require an explicit decision for other findings, and automatically return exceptions for review. The policy needs an owner from the engineering team, not an anonymous spreadsheet maintained by security.

Check the container as the executable artifact it really is

Repository scanning does not replace container scanning. The final image may inherit old system packages, contain a configuration file copied by mistake, include shell utilities, a debug server, or a binary that is absent from the application source.

The check should run after the builder has created the final image and before that digest becomes a deployment candidate. Bind the scan report to the same digest as the SBOM. If the scanner can read an SBOM, that is convenient: you can rescan after updating the vulnerability database without reconstructing the old image. Before adopting this approach, however, test compatibility between the generator, format, and scanner versions in a test pipeline. Even tools from the same ecosystem sometimes change format support out of sync.

One sensible admission sequence looks like this:

1. CI builds the image from pinned input artifacts and publishes it by digest.
2. The generator creates an SBOM for that exact digest, then storage accepts both files as one release set.
3. The scanner checks the image or SBOM against the current database, and policy decides whether to continue.
4. The pipeline signs the image and its build information only after the checks pass.
5. The runtime environment accepts only a digest with a trusted signature and an approved status.

Do not try to put prompt contents, model responses, or API keys into this check. They are not supply-chain components and should not enter either the SBOM or the scan report. Those data types require separate controls: masking, logging policies, access restrictions, and retention periods.

A signature connects the SBOM to a specific build

An SBOM without a connection to the image can easily be replaced or confused with another one. Two files named sbom.json do not prove that one belongs to the other. You need a verifiable chain: a source revision led to a build, the build produced an image with a digest, and that image received a specific SBOM and scan report.

SLSA uses provenance to describe where, when, and how an artifact was created. The practical meaning is simpler than the terminology: after an incident, you must be able to distinguish an image built by your trusted pipeline from the expected revision from an image that someone built locally and uploaded under a similar tag.

A signature proves authorship and integrity only when the runtime environment verifies it. Signing an image in CI while allowing the cluster to download any tag from the registry is pointless. The admission controller or another deployment mechanism should verify at least that:

• the image is specified by digest, not a floating tag;
• the digest has a signature from a trusted publisher;
• the attestation refers to the same digest;
• provenance points to an approved repository and revision;
• the vulnerability status does not prohibit the launch.

Do not trust a version field in a manifest more than a digest. A version is for people, while a digest is for automation. A good release record contains both: a readable name such as 2026.07.23.4 and an immutable content identifier. If the team manually changes a tag after signing, it should create a new release set rather than trying to «reassign» an existing signature.

Build revocation must work during a nighttime incident

Revocation cannot be reduced to deleting a tag from the registry. A running pod has already downloaded the image. An autoscaler may keep the old version. An engineer may have a local copy. Another cluster may use the digest directly. During an incident, you need information about where the release has spread and a sequence of actions approved in advance.

The procedure starts with identification. The on-call engineer receives a CVE, a vendor error report, or a compromise notice. They find the affected component in the SBOM, obtain the list of images containing that component, and then find deployments by digest. If the SBOM cannot be queried by component name and version, you have an archive, not a working tool.

Next, the designated release owner acts. They choose one of three options: temporarily restrict the traffic path, replace the image with a fixed one, or stop the service. For a gateway, simply shutting everything down can be dangerous: business processes may depend on authorization, request processing, or internal assistants. That is not a reason to leave a known risk without a deadline. Temporary risk reduction must have a concrete technical action, such as disabling the vulnerable endpoint, blocking attachment uploads, or restricting access to the internal network.

After replacing the image, verify that the revocation actually took effect, not just that CI reports a successful status:

• whether any affected digest is still present among running workloads in every cluster;
• whether a deployment, job, or cron task still points to the old version;
• whether admission policy blocks another launch of the revoked digest;
• whether the registry has stopped granting permission to publish or deploy this release;
• whether the SBOMs, reports, and decision log were preserved for the incident review.

The most common gap here is organizational, not technical. Without a list of owners, a communication channel, and the authority to stop a release, the on-call engineer collects approvals while the vulnerable gateway continues handling traffic. Run a practice revocation using a safe test digest. There is no need to time people. You need to discover that you lack registry access, cannot see the second cluster, or that admission policy checks only tags.

The checklist must become a release condition

An SBOM works when producing and checking it are part of the normal release path. A separate quarterly task will always lose to an urgent fix, a model migration, and a product deadline.

Check your LLM gateway against this list:

• Every production build has an immutable digest, an SBOM for the final image, a scan report, and a record of the source revision.
• Lockfiles and base images are pinned, and CI does not download unverified dependencies under floating versions.
• Scanning policy distinguishes package presence, reachability, severity, availability of a fix, and temporary exceptions.
• The registry and cluster do not allow a manual launch of an unsigned or revoked digest.
• The team can find every affected release by component and every active deployment by release.

SPDX is useful because it provides a shared language for contents, provenance, and licenses. NIST SSDF is useful because it requires security to be built into the engineering process. But neither a standard nor a scanner will build the decision-making chain for you.

In AI Router, this control is especially relevant to gateways that route requests between external providers and in-house models: the release contents, data-processing boundaries, and change history cannot be reconstructed after the fact. Do not start by buying a new scanner. Start with one image that you can connect today to a digest, an SBOM, a report, and a working revocation procedure. If that cannot be done in one business day, the next incident will make the reason painfully clear.