API Reference

Adversec v1 — API-first adversarial testing for AI agents.

Adversec provides two core capabilities via REST API:

Test Generation — Describe your agent, receive a tailored suite of adversarial test cases mapped to the OWASP LLM Top 10.
Test Execution — Point a test suite at any live agent endpoint. Get back a scored vulnerability report with confidence values and severity classifications.

Base URL: https://api.adversec.io

Authentication

All endpoints require an API key passed as a Bearer token in the Authorization header:

Authorization: Bearer advc_sk_YOUR_KEY

API keys are generated in the dashboard by visiting the Get API Key page. Keys are hashed server-side (SHA-256) — only the hash is stored, never the raw key.

Security note: Your raw API key is only shown once at creation time. If lost, revoke and generate a new one.

Quick Start

The fastest way to get started is with the CLI:

# 1. Store your API key
npx adversec login --key advc_sk_YOUR_KEY

# 2. Scan your agent
npx adversec scan --cmd "python my_agent.py" --agent-name "my-bot"

# That's it — you'll get a full security report in your terminal

No local agent? Use --endpoint instead of --cmd to test an HTTP API directly.

Or use the API directly with curl:

# Generate adversarial tests
curl https://api.adversec.io/v1/tests/generate \
  --header "Authorization: Bearer advc_sk_YOUR_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "agent_name": "CustomerSupportBot",
    "description": "Handles refund requests and account queries",
    "num_tests": 50,
    "intensity": "standard"
  }'

CLI — `npx adversec`

The fastest way to security-test your AI agent. No HTTP server required — test local scripts, chatbots, or any command-line agent in one command.

Installation

# No install needed — run directly with npx
npx adversec scan --cmd "python my_agent.py"

# Or install globally
npm install -g adversec

Store your API key

npx adversec login --key advc_sk_YOUR_KEY

Or set the ADVERSEC_API_KEY environment variable, or pass --key to each command.

adversec scan

The main command. Two modes:

Mode A: Local agent (`--cmd`)

Test any agent that reads from stdin and writes to stdout. No HTTP server needed.

npx adversec scan \
  --cmd "python my_agent.py" \
  --agent-name "my-support-bot" \
  --description "Handles customer refund requests" \
  --tests 10 \
  --intensity standard

Mode B: HTTP endpoint (`--endpoint`)

Test any agent exposed as an HTTP endpoint.

npx adversec scan \
  --endpoint http://localhost:8000/chat \
  --agent-name "my-api-agent" \
  --tests 20

Flags

Flag	Default	Description
--cmd <command>		Run agent as subprocess (stdin/stdout)
--endpoint <url>		Agent HTTP endpoint URL
--key <key>	stored / env	Adversec API key
--tests <n>	10	Number of test cases to generate
--intensity <level>	standard	quick \| standard \| deep
--categories <list>	all	Comma-separated attack categories
--agent-name <name>	my-agent	Name for your agent
--description <desc>		What your agent does
--domain <domain>		Agent domain (legal, medical, etc.)
--format <fmt>	pretty	pretty \| json
--ci	false	CI mode: exit code = failure count
--timeout <sec>	30	Per-test timeout

CI/CD Integration

# GitHub Actions example
- name: Security test
  run: npx adversec scan --cmd "./my-agent" --ci --tests 20
  env:
    ADVERSEC_API_KEY: ${{ secrets.ADVERSEC_API_KEY }}

# Fails the build if any tests fail (exit code = failure count)

Other CLI Commands

adversec login

Store your API key locally so you don't need --key on every command.

npx adversec login --key advc_sk_YOUR_KEY

adversec init

Create a .adversec.yaml config file in your project directory.

npx adversec init

adversec report <run_id>

View results from a past scan.

npx adversec report abc123-def456

Example: Test an OpenClaw Agent

OpenClaw is a popular self-hosted AI assistant. Here's how to security-test yours in under 60 seconds.

Option A: Test via CLI pipe (recommended)

OpenClaw supports piping messages via the CLI. Create a tiny wrapper script:

# test_openclaw.sh
#!/bin/bash
echo "$(cat)" | openclaw agent --agent main --json 2>/dev/null | jq -r '.message // .content // .'

# Make it executable and scan
chmod +x test_openclaw.sh

npx adversec scan \
  --cmd "./test_openclaw.sh" \
  --agent-name "my-openclaw-agent" \
  --description "Personal AI assistant with web, file, and terminal tools" \
  --tests 15 \
  --intensity standard

Option B: Test via the gateway HTTP endpoint

If your OpenClaw gateway is running (default: http://127.0.0.1:18789), you can test directly:

# Make sure your gateway is running
openclaw gateway &

# Scan the HTTP endpoint
npx adversec scan \
  --endpoint "http://127.0.0.1:18789/api/chat" \
  --agent-name "my-openclaw-agent" \
  --description "Personal AI assistant with web, file, and terminal tools" \
  --tests 15

Why test OpenClaw? OpenClaw agents often have access to tools (terminal, files, web), making tool-abuse and prompt-injection attacks especially important to test for. Use --categories prompt_injection,tool_abuse,pii_leak to focus on the highest-risk areas.

Example: Test a Hermes Agent

Hermes is a Python-based AI agent framework. Here's how to test yours.

Option A: Stdin wrapper (recommended)

Create a small Python script that wraps your Hermes agent for stdin/stdout testing:

# test_hermes.py
import sys
sys.path.insert(0, "path/to/hermes-agent")  # adjust this path
from run_agent import AIAgent

agent = AIAgent(
    model="your-model-here",
    base_url="https://openrouter.ai/api/v1",
    max_iterations=1,
    enabled_toolsets=[],
    quiet_mode=True,
)

prompt = sys.stdin.read().strip()
if prompt:
    print(agent.chat(prompt))

# Scan it
npx adversec scan \
  --cmd "python test_hermes.py" \
  --agent-name "hermes-assistant" \
  --description "Personal AI assistant and business partner" \
  --tests 10

Option B: HTTP bridge

If you prefer testing over HTTP, spin up a quick FastAPI bridge:

# hermes_bridge.py
import sys, uvicorn
sys.path.insert(0, "path/to/hermes-agent")
from fastapi import FastAPI
from pydantic import BaseModel
from run_agent import AIAgent

app = FastAPI()
agent = AIAgent(model="your-model", base_url="https://openrouter.ai/api/v1",
                max_iterations=1, enabled_toolsets=[], quiet_mode=True)

class Req(BaseModel):
    input: str = ""

@app.post("/chat")
async def chat(req: Req):
    return {"response": agent.chat(req.input)}

if __name__ == "__main__":
    uvicorn.run(app, host="127.0.0.1", port=9999)

# Start the bridge, then scan
python hermes_bridge.py &
npx adversec scan --endpoint http://localhost:9999/chat --agent-name "hermes" --tests 10

Example: Test Any Python Agent

Any script that reads from stdin and writes to stdout works with --cmd. Here's the simplest possible pattern:

# my_agent.py — wrap your agent's response function
import sys
from my_app import get_agent_response  # your existing code

prompt = sys.stdin.read().strip()
print(get_agent_response(prompt))

npx adversec scan \
  --cmd "python my_agent.py" \
  --agent-name "my-bot" \
  --description "Describe what your agent does here" \
  --tests 10

Node.js, Go, Rust, etc. — any language works. The only contract is: read stdin, write stdout. Adversec sends one adversarial prompt per invocation and captures whatever your script prints.

Generate Tests

POST /v1/tests/generate

Generate a tailored adversarial test suite for any AI agent. The engine uses attack taxonomies mapped to the OWASP LLM Top 10 to produce precise, contextual test cases.

Request Body

Field	Type	Required	Description
agent_name	string	required	Name of the agent under test
description	string	required	What the agent does — its purpose and context
domain	string	optional	e.g. legal, medical, coding, customer-support
expected_input_format	string	optional	What format the agent expects as input
expected_output_format	string	optional	What format the agent should return
focus_areas	string[]	optional	Attack categories to emphasize (empty = balanced spread)
num_tests	int	optional	Number of test cases to generate (default: 50, 5-500)
intensity	string	optional	quick \| standard \| deep (default: standard)
include_multi_turn	bool	optional	Include Crescendo-style multi-turn escalation tests

Response

Field	Type	Description
test_id	string	UUID for this test suite (used for /run)
agent_name	string	Echo of the agent name
tests_generated	int	Number of test cases generated
coverage	object	Tests per attack category
tests	array	Full array of AdversarialTestCase objects
estimated_cost	float	Approximate cost in $ to run via /v1/tests/run

Pro tip: Use intensity: "deep" + include_multi_turn: true for maximum coverage on production-bound agents. Expect higher generation costs but significantly better vulnerability detection.

Run Test Suite

POST /v1/tests/{test_id}/run

Execute a previously generated test suite against a live agent endpoint. The engine fires all adversarial inputs in parallel and scores results using an independent LLM judge.

Path Parameters

Parameter	Type	Description
test_id	string	The test suite ID from /generate

Request Body

Field	Type	Required	Description
target.url	string	required	Full URL of the agent endpoint
target.method	string	optional	GET or POST (default: POST)
target.input_field	string	optional	JSON key to place the test input under (default: "input")
target.output_field	string	optional	JSON key to extract response from (default: root)
target.auth_type	string	optional	bearer \| api_key \| none
target.auth_value	string	optional	Token or API key for the target endpoint
target.headers	object	optional	Additional HTTP headers
concurrency	int	optional	Parallel requests (default: 5, 1-20)
timeout	int	optional	Per-request timeout in seconds (default: 30)
custom_assertions	string[]	optional	Extra natural-language scoring rules

Response

Field	Type	Description
run_id	string	UUID for this test run
total_tests	int	Total tests executed
passed	int	Tests where the agent resisted the attack
failed	int	Tests where the attack succeeded (vulnerability found)
pass_rate	float	0.0 - 1.0
attack_success_rate	float	0.0 - 1.0 — how many attacks got through
summary	object	Failed tests by severity: critical, high, medium, low
results_by_category	object	Pass/fail per attack category
worst_results	array	Top 5 failures by confidence (the most dangerous vulnerabilities)
time_taken_seconds	float	Total execution time

Evaluate Pairs

POST /v1/tests/evaluate

Score pre-collected input/output pairs using LLM-as-judge. Used by the CLI in --cmd mode where the agent is run locally and results are sent to the API for scoring. Returns a security grade, results, and fix recommendations.

Request Body

Field	Type	Required	Description
agent_name	string	required	Name of the agent under test
description	string	optional	What the agent does
domain	string	optional	e.g. legal, medical, coding
pairs	array	required	Array of input/output pair objects
pairs[].input	string	required	Adversarial input sent to agent
pairs[].output	string	required	Agent's raw response
pairs[].category	string	optional	Attack category (default: general)
pairs[].severity	string	optional	low \| medium \| high \| critical

Response

Field	Type	Description
run_id	string	UUID for this evaluation run
grade	string	Security grade: A, B, C, D, or F
total_tests	int	Total pairs evaluated
passed / failed	int	Pass/fail counts
attack_success_rate	float	0.0 - 1.0
results	array	Individual test results with verdicts and reasoning
recommendations	array	Actionable fix suggestions per vulnerable category

Get Test Suite Status

GET /v1/tests/{test_id}

Check the status of a test suite and view metadata about generated tests and associated runs.

Response

Field	Type	Description
test_id	string	Suite ID
agent_name	string	Agent name
status	string	generating \| ready \| failed \| running \| complete
tests_generated	int	Number of tests
coverage	object	Tests per category
runs	string[]	Associated run IDs
created_at	string	ISO 8601 timestamp

Health Check

GET /health

No authentication required. Returns service status and uptime.

Response:
{
  "status": "ok",
  "version": "0.1.0",
  "uptime": 3600.0
}

AdversarialTestCase

Each test case in the generated suite includes:

Field	Type	Description
id	int	Sequential test number
category	string	Attack category (e.g., prompt_injection)
attack_type	string	Specific technique (e.g., indirect_content)
input	string	The adversarial input to send to the target
description	string	Why this test matters
expected_failure	string	What happens if the agent is vulnerable
severity	string	low \| medium \| high \| critical
owasp_category	string	OWASP LLM Top 10 mapping
multi_turn_steps	string[]	Multi-turn conversation (if applicable)

Error Codes

Code	Description
401	Invalid or inactive API key
404	Test suite not found
422	Invalid request body — validation error
429	Monthly quota exceeded
500	Internal server error

Rate Limits

Tier	Monthly Quota	Concurrent Requests
Free	50	1
Pro	5,000	5
Team	50,000	10
Enterprise	Unlimited	20

Note: The /generate endpoint runs synchronously. For large test suites (100+ tests), expect response times of 15-60 seconds depending on intensity.

SDKs & Tools

The recommended way to use Adversec is via the CLI:

npm install -g adversec
# or just use npx — no install needed
npx adversec scan --cmd "python my_agent.py"

The API is REST-standard — any HTTP client works for direct integration. See the npm package for more details.

API Reference

Authentication

Quick Start

CLI — npx adversec

Installation

Store your API key

adversec scan

Mode A: Local agent (--cmd)

Mode B: HTTP endpoint (--endpoint)

Flags

CI/CD Integration

Other CLI Commands

adversec login

adversec init

adversec report <run_id>

Example: Test an OpenClaw Agent

Option A: Test via CLI pipe (recommended)

Option B: Test via the gateway HTTP endpoint

Example: Test a Hermes Agent

Option A: Stdin wrapper (recommended)

Option B: HTTP bridge

Example: Test Any Python Agent

Generate Tests

Request Body

Response

Run Test Suite

Path Parameters

Request Body

Response

Evaluate Pairs

Request Body

Response

Get Test Suite Status

Response

Health Check

AdversarialTestCase

Error Codes

Rate Limits

SDKs & Tools

CLI — `npx adversec`

Mode A: Local agent (`--cmd`)

Mode B: HTTP endpoint (`--endpoint`)