497 real attacks on a public, reproducible benchmark

Your agent acts with your credentials. Decide what it's allowed to do.

A poisoned document can steer the agent you shipped, silently, with the access you already gave it. Crawdad is a transparent local proxy between your agent and its model: it inspects every request and bounds what each agent may do by its purpose, judged on the action it takes rather than the intent it claims. One env var, no SDK, no code changes, and raw content never leaves your machine by default. Building the platform others run their agents on? Embed detection directly with the gateway /v1 API.

terminal
$ curl -fsSL https://getcrawdad.dev/install.sh | sh
Crawdad Dashboard, real-time detection overview, threat categories, agent activity

Running in seconds.

Install, point your agent's base URL at the local proxy, activate your license. Crawdad intercepts your agent's provider traffic transparently, your agent doesn't know it's there.

  • One install command, macOS, Linux, WSL
  • Route via ANTHROPIC_BASE_URL or equivalent, no code changes
  • Observe-first by default: records and scores, does not block until you arm it
  • Free tier includes the full detection pipeline, no capability gating
quickstart
# 1. Install
$ curl -fsSL https://getcrawdad.dev/install.sh | sh

# 2. Route your agent through Crawdad (Anthropic port, no /v1)
$ export ANTHROPIC_BASE_URL=http://localhost:7748

# 3. Activate
$ crawdad setup --license

# That's it. Every request is now inspected.

Protect your agent, or embed detection in your product.

The same detection engine, two integration shapes. Run it in front of your own agent for the full on-device pipeline, or call the hosted gateway API to add detection to the product you're building for others.

Proxy path · one env var

Run it in front of your agent

Point your agent's base URL at the local proxy. You get the full multi-layer ML pipeline running inline and on-device, the engine behind the public benchmark, plus charters, trajectory reasoning, and, when you opt in, OS-level Enforce. No SDK, no code changes, and raw content never leaves the machine.

  • The full on-device detection pipeline, ML on every input
  • Content stays on-device: prompts and responses stay local
  • Best for: securing an agent you run
Embed path · the /v1 API

Call it from your product

Send content to the hosted gateway POST /v1/detect with a scoped API key and get a decision back. It runs a fast pattern-and-heuristic engine, purpose-built for embedding a detection check into your own platform, agents, keys, policy, and audit under one versioned REST surface.

  • Scoped API key, OpenAPI 3.1, interactive reference
  • Decision, threat score, categories, reason
  • Best for: platform owners adding detection

One honest distinction, stated up front. The detection benchmark belongs to the local sidecar's full ML pipeline on the proxy path. The gateway /v1/detect endpoint is the faster pattern-and-heuristic engine, a different, lighter surface, so we never attach the sidecar's benchmark number to it. Pick the path that fits, and know exactly what each one runs.

Charters: bound an agent in a policy file.

Detection inspects the request. A charter bounds what the agent may do with it. Declare the agent's real job as an allowlist and Crawdad enforces it on the observed action, the tool call the agent actually makes, so a deceptive stated intent buys nothing.

A charter is three axes: the tools the agent may call, the data (paths, hosts, recipients) it may touch, and the effects it may produce. Author it in your KDL policy file, and on reload it governs at the wire on the next action, with no sidecar restart. Anything outside the charter is blocked at the wire, and the block names the axis it violated.

  • Enforced on the observed action, not the stated intent
  • A block names the axis: capability, target, or effect
  • The same request from two agents can get two verdicts
  • Opt-in and additive, a policy with no charter behaves exactly as before
policy.kdl
// A reporting agent may read the reports tree
// and email the reporting domain, nothing else.
policy "reporting-agent" {
  identity { agent "report-*" }
  charter {
    tools "Read" "Grep" "Glob" "send_email"
    data "/reports/**" "*@acme-reports.com"
    effects "read" "external-send"
  }
}

Read /reports/q3.txt is in-charter here, out-of-charter for a support-triage agent.

Trajectory

The whole session, not one step

Enumerate, read progressively more sensitive in-scope data, then send. Every step is individually allowed; the shape is the attack. Crawdad scores that shape with deterministic signals anchored to resource sensitivity and gates an on-device reasoner onto the ambiguous band. With no local model reachable, a completed staging chain is held for review, never silently allowed. Zero false positives measured on benign multi-step work. It does not claim to catch every composed harm; it catches staged compromise that carries a sensitivity climb.

Demo

Drive the real enforcement path

A runnable demo (crawdad demo-serve) drives the production proxy with the real charter, trajectory, and session-risk engines. The verdicts are computed by the production engine; the scenario is scripted so it reproduces and tool side effects are simulated; free-drive returns a real verdict on any action you supply.

Multi-layer inspection. Every request.

Not a single classifier. Not a regex list. A multi-layer prompt injection protection pipeline where each layer catches what the others miss. Built in Rust. 3,737 tests across 26 crates.

Request In agent → proxy
L1 Patterns 127 / 22 categories
Behavioral 7 sub-checks
DeBERTa 44M classifier
Arbiter final verdict
Model or blocked
L1

Pattern Matching

128 patterns across 22 attack categories. Catches known injection formats, encoding tricks, and role-override attempts before anything else runs.

128 patterns · 22 categories
Behavioral

Behavioral Analysis

7 sub-checks analyze structural signals, conversation topology, authority escalation, instruction-data boundary violations. Catches attacks that look benign to static patterns.

7 sub-checks
Indirect

Indirect Injection

Detects instructions hidden inside documents, tool outputs, and retrieved content. The threat that enters through what your agent reads, not what the user types.

Session

Session Context

Maintains a 20-message sliding window to catch multi-turn attacks that unfold gradually, slow escalation, context poisoning, progressive boundary shifting.

20-message window
PII / Cred

Credential & PII Detection

Scans for 10 credential and PII types, API keys, tokens, SSNs, credit cards, before they reach the model. Blocks exfiltration at the proxy layer.

10 credential types
ML

DeBERTa-v2-small Classifier

44M-parameter transformer fine-tuned on adversarial prompt data. Runs locally on-device. Catches novel attacks that evade pattern and behavioral layers.

DeBERTa-v2-small · 44M params

The Arbiter combines signals from every layer into a single verdict, then decides what to block. The credential and data-exfiltration floors always block inline, and a single high-confidence detection is promoted to an inline block, so on the proxy path blocking tracks detection with a low false-positive rate.

POST /v1/detect
POST /v1/keys
GET /v1/agents
POST /v1/policy/rules
GET /v1/audit/events
DEL /v1/keys/{key_id}

Full reference in Redoc →

The gateway /v1 surface. Documented in Redoc.

For platform builders: a clean, versioned REST surface to embed detection and manage agents, keys, policy, and audit from your own product. POST /v1/detect runs the fast pattern-and-heuristic engine and returns a decision, threat score, categories, and reason. Authenticate with a scoped API key; every path is in the OpenAPI 3.1 spec and the interactive reference.

  • Scoped API key auth (Authorization: Bearer or X-API-Key)
  • Detection, agents, policy, audit, and fleet status
  • OpenAPI 3.1 spec + interactive Redoc reference
  • Fast pattern engine, not the sidecar's full ML pipeline, and we say so
API Reference → Contact Us for API Access

Works with what you already use.

Crawdad is a transparent proxy, an LLM firewall that sits between your agent and the model. If your tool talks to an LLM API, Crawdad can inspect it. No SDK, no code changes.

Agents & Tools
Claude Code Anthropic CLI
Windsurf AI IDE
Cline VS Code Extension
Aider Terminal Agent
Continue IDE Extension
Providers
Anthropic Claude
OpenAI GPT / o-series
Google Gemini
xAI Grok
NVIDIA NIM
497 real attacks on a public, reproducible corpus
497attacks tested
1,669total samples
22attack categories
CC BY 4.0corpus license
Every result published, including the one miss and the one false positive. Clone the corpus, run the benchmark, compare your tool. Every number is reproducible.

497 attack samples, 1,172 benign negatives, 22 categories. The entire corpus is open under CC BY 4.0. The product is licensed under BSL 1.1.

AndrewSispoidis/contemporary-agent-attacks →

Local-first. Content stays on-device.

Crawdad runs as a transparent proxy on your machine. Raw content, prompts, responses, tool outputs, never leaves your device by default. Metadata-only telemetry by default, customer-governed.

Your machine, raw content stays local
🤖
Your Agent
any framework
Crawdad Proxy
inspects every request
🧠
AI Provider
any model
✓ on-device detection ✓ raw content stays local ✓ transparent to your agent ✓ one env var to enable

One environment variable routes your agent's traffic through the local proxy. Your agent doesn't need to know Crawdad exists. Detection runs entirely on-device.

Enforce mode · new in 1.6, opt-in

Ready to make the proxy the only way out?

By default Crawdad runs in Monitor mode, exactly the transparent behavior above: a stock device is byte-for-byte unchanged and the env var is a door your agent chose to walk through. When you want a harder boundary, opt a device into Enforce and the operating system installs a persistent default-deny egress lock (pf on macOS, iptables on Linux) so a governed agent reaches only the Crawdad path, everything else, including UDP/QUIC, denied by construction and the lock persisting across a kill. Full runtime enforcement runs on macOS and Linux today. Read the Enforce mode guide →

Ship agents your users can actually trust.

Start with the install. Read the docs. Run the benchmark yourself.

Get in Touch

Tell us about your needs and we'll follow up.