Shield as an LLM gateway guardrail

An LLM gateway sits between your agents and the model provider. Agents send it their requests with a key you issue, and it forwards them with a provider key that only the gateway holds. An agent uses the gateway through its base-URL setting; for Claude Code, that is one environment variable. Shield plugs into the gateway as a response guardrail: every response on the gateway's Messages route passes through Shield before the gateway returns it to the agent.

The design fits any gateway that meets the requirements below. Our reference deployment is LiteLLM, an open-source gateway, and it is the only gateway we have tested. The set-up, the error developers see and the test results on this page are LiteLLM's.

Pilot

Tested with LiteLLM 1.105.0, Claude Code 2.1.296, and a mock of the Anthropic Messages API. The LiteLLM guardrail module and the shield-server build come with the pilot. Another gateway needs its own adapter, which a pilot builds and tests against the requirements below. Live-provider testing is the pilot's first step. See what the pilot covers.

Zoomable version: open the interactive gateway architecture.

How it works

1. The agent sends its normal request to your gateway. It holds a gateway key that you issue, not a provider key.
2. The gateway checks that key and forwards the request to the provider with the provider key, which only the gateway holds.
3. The provider streams its response back. The Shield guardrail holds the whole stream: nothing goes to the agent yet. It reassembles every tool call, including arguments that arrive in fragments and several calls in one response.
4. The guardrail sends each tool call to the Shield server. A shell command is checked as a command; any other tool is checked by name and arguments, exactly as the AgentShield hook does.
5. If every tool call is approved, the guardrail releases the provider's original bytes, unchanged. Otherwise the agent receives an HTTP 400 and no tool call.

The stream parser, the tool-call mapping and the release rule are shared code, the same in both gateway options. Each adapter does the rest: it frames the stream, parses a non-streamed response, applies the size and time limits, and connects them to the gateway's hooks.

What a gateway needs

The guarantees on these pages rest on six properties of the gateway. Each one below says how LiteLLM provides it in the reference deployment, some only with the settings shown further down, and where it falls short or is untested. When you evaluate another gateway, check each one.

• A hook that holds each response, streamed or not, before any of it is delivered. A hook that runs after the stream has reached the agent can record a tool call but cannot stop it.

  LiteLLM: for a streamed response, the guardrail wraps LiteLLM's streaming response hook; for a non-streamed one, it uses the post-call hook, which runs before delivery. With the guardrail configured as below, LiteLLM's post-call hook cannot hold a stream: on its chat-completions route it runs only after a stream has been delivered and only logs an error, and on the Messages route it does not run for streams at all.

• The hook can replace the response with an error the agent shows. That is how a block reaches the developer. Claude Code reports a 403 as an authentication failure, so the guardrail returns a 400.

  LiteLLM: the guardrail raises LiteLLM's guardrail error with status 400.

• A failure inside the guardrail withholds the response. A gateway that logs a failed guardrail and delivers the response anyway fails open.

  LiteLLM: the guardrail turns the failures it anticipates (a Shield outage, an invalid verdict, a malformed, truncated or oversized response) into that 400, and each is tested. How LiteLLM handles an error the guardrail does not catch is not tested.

• A request cannot switch the guardrail off. Otherwise any key holder can opt out.

  LiteLLM: a request can opt out of a default guardrail through metadata. The Shield guardrail ignores that, and a check confirms that a request carrying the opt-out is still blocked.

• Every route that reaches the provider runs the guardrail. Gateways often serve other routes, such as pass-through or other API formats, that skip guardrails. Each one is a way around Shield, and so is a response cache that serves stored responses without running the hook. List the routes your gateway serves.

  LiteLLM: its /anthropic/* pass-through skips guardrails. The warning below gives the settings that close it to gateway keys, and the gap that remains.

• The gateway holds the provider key and knows each user. Developers then need no provider key of their own, and each record names a user.

  LiteLLM: virtual keys tied to a user id. The guardrail sends that user to Shield. Not yet tested: the lab used the master key.

The guardrail reads responses in the Anthropic Messages format, streamed or not. A gateway route that serves another API format needs its own parser, which the pilot has not built, and that route must meet the six properties above. Adding an agent other than Claude Code also takes the rest of which agents it covers, including the mapping of its shell tool.

Setting it up: the LiteLLM reference

The pilot runs on one host: LiteLLM and the Shield server side by side. Shield listens on loopback only, so it is not reachable from the network.

1. Run the Shield server in enforce mode, with a random token that LiteLLM will share:

export AGENTSHIELD_SERVER_TOKEN=$(openssl rand -hex 32)
mkdir -p /var/log/shield
shield-server --addr 127.0.0.1:8383 --mode enforce --log /var/log/shield/audit.jsonl

The server does not create the log directory. If it cannot open the log, it still starts and answers, and warns that evaluations will not be recorded.

2. Add the guardrail to the LiteLLM config. In LiteLLM's environment, set SHIELD_URL=http://127.0.0.1:8383 and SHIELD_TOKEN to the same token, and put the pilot's shield_guardrail.py and response_gate.py on its Python path.

model_list:
  - model_name: claude-sonnet-4-5-20250929        # see "Model names" below
    litellm_params:
      model: anthropic/claude-sonnet-4-5-20250929
      api_key: os.environ/GATEWAY_ANTHROPIC_KEY   # NOT ANTHROPIC_API_KEY: see the warning below

guardrails:
  - guardrail_name: shield
    litellm_params:
      guardrail: shield_guardrail.ShieldGuardrail
      mode: post_call
      default_on: true

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Keep the provider key out of LiteLLM's default variable names

LiteLLM also serves a built-in pass-through route, /anthropic/*, that skips guardrails. It signs requests with the gateway's ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN, or with a model marked use_in_pass_through: true. A gateway set up that way lets any key holder reach the provider without Shield, just by pointing Claude Code at <gateway>/anthropic. So:

• store the provider key under another name, as above, and never set ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN on the gateway host;
• never set use_in_pass_through: true on a model.

With no credential under those names, LiteLLM refuses a pass-through request that carries only a gateway key. A check in our lab confirms that it delivers no tool call, and a control run shows a gateway holding ANTHROPIC_API_KEY does leak the call.

LiteLLM still relays, unguarded, a pass-through request that brings the developer's own Anthropic API key or sign-in token. That is the same gap as a developer with their own provider key going direct. But while this route is reachable, blocking direct egress to the provider does not close it. To close it, put a proxy in front of LiteLLM that forwards only POST /v1/messages and its token-count endpoint. Our lab pins the current behaviour with a check, and the front proxy is pilot work: it is not yet tested.

3. Point the agents at the gateway on each developer machine. For Claude Code:

export ANTHROPIC_BASE_URL=https://llm-gateway.example.internal
export ANTHROPIC_API_KEY=<gateway key issued to this developer>

Model names. LiteLLM routes by model name, and Claude Code sends real model ids, so name each model_list entry after an id your Claude Code version sends, or pin the laptops to your gateway's names with ANTHROPIC_MODEL. Claude Code also calls a smaller background model, for example to summarise fetched pages; give that model a gateway entry too, or override it. A request for a name the gateway does not know gets a 400. Our lab pinned the model on the command line.

Keys. Do not give developers LiteLLM's master key: it is an administrator credential for the whole gateway. Issue each developer a LiteLLM virtual key tied to a user id, which needs LiteLLM's database. The guardrail records that user as the principal on every Shield decision. A key with no user id is recorded under a placeholder, and one shared key makes every record name the same user. The pilot lab used the master key, so per-developer keys are part of the pilot.

What developers see

An approved response is indistinguishable from the provider's own. A blocked one stops the turn with an API error that names the rule. Shield's part of the message is Shield: blocked action; rules=<rule id>, and the gateway adds its own wrapper. On LiteLLM it reads:

API Error: 400 Guardrail raised an exception, Guardrail: shield,
Message: Shield: blocked action; rules=<rule id>

The response is withheld whole. If the model proposed three tool calls and one is blocked, none of them run. HTTP 400 was chosen over 403 because Claude Code reports a 403 as an authentication failure, which would point developers at the wrong problem. (Our tests turn Claude Code's automatic retries off, so how many times a default client retries a 400 is not yet measured.)

What we tested (LiteLLM)

Each check ran on 14 October 2026, on the LiteLLM reference deployment, against Shield main plus pilot branch PR #4354. "Real Claude Code" means the CLI was run end to end, and the result is judged by files on disk and what reached the provider, not by logs.

Case	Result
Approved shell command, streamed	Real Claude Code runs it
Blocked shell command	HTTP 400 naming the rule; real Claude Code runs nothing
Two tool calls in one response, one blocked	Neither runs
Claude Code's own Read tool, approved	Released; the file's content reaches the provider in the next turn
Claude Code's own Read tool, blocked by MCP policy	HTTP 400 naming the rule; the file is never read
Write tool, approved and blocked, streamed and non-streamed	Released or withheld as Shield decided
Shield server stopped	HTTP 400; real Claude Code runs nothing
Shield degraded, in audit-only mode, asking for approval, or returning an unknown verdict	Nothing released
Malformed, truncated or oversized response; undeclared tool; provider-run tool	HTTP 400, no tool call
A shipped Shield rule (deleting the filesystem root)	Blocked
Request asks LiteLLM to skip the guardrail	Still blocked
Request to LiteLLM's /anthropic/* pass-through carrying only the gateway key, gateway set up as above	Refused; no tool call, streamed or not
The same route, with the developer's own provider key added	Relayed unguarded: a known gap, pinned by a check (see the warning above)
Request for a model the gateway does not list	HTTP 400

In observe mode, with an audit-only Shield:

Case	Result
A call Shield would block	Released, and recorded as AUDIT with the original decision BLOCK; real Claude Code runs it
Shield server stopped	Released, and logged by the gateway as unevaluated

That is 30 end-to-end checks, one of which pins the known gap above, and 49 checks of the guardrail on its own. The lab used LiteLLM's master key and ran Claude Code non-interactively.

Shield's decision records are written on a best-effort basis in the pilot: if the audit log cannot be written, Shield still answers. Monitor the log; durable delivery is pilot work.

Limits to plan for

• An agent that does not use the gateway is not covered. A developer who has their own provider key can point their agent straight at the provider, or, on LiteLLM, at the gateway's /anthropic/* route. Keep provider keys off laptops, and keep the AgentShield hook installed. For enforcement that does not depend on each agent's gateway URL, see network inspection.
• Only the Messages route is guarded. Do not expose other gateway routes to developers' keys if they could reach the provider. On LiteLLM, also keep the provider key out of its default variable names (see the warning above).
• Coverage is the hook's coverage. A tool is only as well covered as the policy that matches its name and arguments. For example, a Read of an SSH private key is blocked, while a Write to a system file is recorded, not blocked, through the gateway and the hook alike. Managed-mode protections that guard AgentShield's own files on a laptop do not apply at the gateway.
• Response size and time. The LiteLLM guardrail holds up to 2 MiB and 60 seconds per response, and each step of a Shield call (connect, send, read) has a 5-second limit. Anything beyond those is withheld. The size limit is tested; the two time limits are not yet exercised by a check.
• One gateway, one organisation. The pilot is single-tenant, and records stay on the gateway.