Skip to main content

Skill-script integrity

Preview

Each half is tested in its own product, and the whole flow below was run end to end on 2026-10-13. There is no automated test across the two products yet, and the manifest is handed over by hand.

Agent skills (SKILL.md packages) often bundle scripts the agent runs on your behalf. You may have reviewed a skill when you adopted it. That review means nothing if one of its scripts is changed afterwards: the agent keeps running it under the skill's trust. That is a skill rug pull.

This feature closes that gap in two steps:

  1. Comply records what you reviewed. It reads the skills in a repository and writes a manifest with the SHA-256 of every bundled script.
  2. AgentShield checks it at run time. When the agent runs one of those scripts, AgentShield hashes the file on disk and compares it with the manifest. A mismatch is recorded, or blocked if you choose. The threat category is supply-chain/artifact-integrity/skill-script-tampering.

1. Write the manifest with Comply​

Comply is available on paid plans. Run it in the repository whose skills you have reviewed:

mkdir -p .comply
docker run --rm --platform linux/amd64 -v "$PWD":/scan ghcr.io/ai-agentlens/comply:latest \
  generate-policy --path /scan \
  --taxonomy /usr/local/bin/taxonomy --compliance /usr/local/bin/compliance \
  --output /scan/.comply/mcp-policy.yaml --artifacts-output /scan/.comply/artifacts.yaml

The manifest lists each script, its hash and the skill it belongs to:

version: 1.0.0
artifacts:
    - path: .claude/skills/pdf-tools/scripts/extract.py
      sha256: 0aea40ea6a9661779fc620c045e2d3503a4b48386d8ea4833932a743d9896f5f
      skill: pdf-tools
      source: .claude/skills/pdf-tools/SKILL.md

The same run also writes --output, an AgentShield MCP policy pack for the repository; see From scan to runtime policy. Commit the manifest, or distribute it the way you distribute other reviewed configuration.

2. Point AgentShield at it​

The check is off until a manifest is configured, and when it is off it costs nothing:

export AGENTSHIELD_ARTIFACT_MANIFEST=/path/to/repo/.comply/artifacts.yaml
export AGENTSHIELD_ARTIFACT_VERIFY_MODE=audit   # or: block

Set these in the environment the agent runs in. Start with audit, look at what it records, then move to block.

What it does, precisely​

  • When a command runs a script listed in the manifest, AgentShield resolves the path against the working directory. It also matches a repository-relative suffix, so the check still works when the agent runs from a subdirectory.

  • It only reports a confirmed mismatch: the file is readable and its hash differs from the manifest. A missing file, an unreadable file or a path it cannot line up is skipped, so the check never blocks on uncertainty.

  • A mismatch is reported as rule comply-skill-artifact-tamper, with both short hashes in the reason. It is AUDIT by default and BLOCK in block mode:

    BLOCK
      Rules: comply-skill-artifact-tamper
      Reason: Skill bundled script '.claude/skills/pdf-tools/scripts/extract.py' does not match
              its declared integrity hash (declared 0aea40ea6a96…, on disk 1b51b4f7843a…). …

  • When you change a script on purpose, run Comply again to refresh the manifest.