This guide walks through installation and every usage mode step by step.
| Requirement | Version | Notes |
|---|---|---|
| Python | 3.10+ | python.org |
| Node.js | 18+ | Only for VS Code extension build |
| VS Code | Any | Only if using the extension |
| Claude Desktop | Any | Only if using MCP integration |
git clone https://github.com/aparnaa19/agentforensics
cd agentforensics
python install.py
This single command:
pip install agentforensics[dashboard]
If you skipped the model during install:
python scripts/download_model.py
The dashboard is the central hub - all detections appear here regardless of which integration you use.
Option A - Desktop app (Windows)
dist/AgentForensics.exeOption B - Command line (all platforms)
python -m agentforensics.cli dashboard
Option C - Python
from agentforensics.dashboard import start
start()
The dashboard opens at http://localhost:7890
To change the port:
python -m agentforensics.cli dashboard --port 8080
Monitor any Python app that uses the OpenAI or Anthropic SDK. No other code changes needed.
from agentforensics import trace
from openai import OpenAI
# Wrap the client - one line change
client = trace(OpenAI(api_key="sk-..."))
# Use exactly as normal
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Summarise this document: ..."},
],
)
print(response.choices[0].message.content)
from agentforensics import trace
from anthropic import Anthropic
client = trace(Anthropic(api_key="sk-ant-..."))
response = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
messages=[{"role": "user", "content": "Summarise this email: ..."}],
)
session_id is printed to console on first calltracer = trace(client)
print(tracer.session_id) # e.g. "abc123-..."
tracer = trace(client)
@tracer.on_injection
def handle(signal):
print(f"Injection detected! Score: {signal.score:.2f}")
print(f"Evidence: {signal.evidence_snippet}")
AgentForensics plugs directly into Claude Desktop as an MCP server. Every conversation Claude has is monitored - no code changes, no API keys needed.
Run python install.py - it configures MCP automatically.
If you want to configure manually, add this to your Claude Desktop config (claude_desktop_config.json):
{
"mcpServers": {
"agentforensics": {
"command": "python",
"args": ["/path/to/agentforensics/agentforensics/mcp_server.py"]
}
}
}
Config file locations:
%APPDATA%\Claude\claude_desktop_config.json~/Library/Application Support/Claude/claude_desktop_config.jsonpython -m agentforensics.cli dashboardThe extension automatically scans every file you open, edit, or paste in VS Code. Injections are shown as red squiggles and in the Problems panel.
install.py), or find the .vsix file in agentforensics-vscode/Extensions (Ctrl+Shift+X) → ⋯ menu → Install from VSIX.vsix fileCtrl+Shift+M) to see all detectionsAgentForensics: CLEAN / SUSPICIOUS / COMPROMISEDScan with AgentForensicsGo to VS Code Settings → search “AgentForensics”:
| Setting | Default | Description |
|---|---|---|
agentforensics.dashboardUrl |
http://localhost:7890 |
Dashboard address |
agentforensics.scanOnOpen |
true |
Scan files when opened |
agentforensics.scanOnPaste |
true |
Scan multi-line pastes |
Monitor LangChain agent runs with a callback handler.
pip install langchain langchain-openai
from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain_openai import ChatOpenAI
from agentforensics.integrations.langchain_handler import AgentForensicsHandler
llm = ChatOpenAI(model="gpt-4o-mini")
agent = create_openai_tools_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)
# Add the handler
result = agent_executor.invoke(
{"input": user_input},
config={"callbacks": [AgentForensicsHandler(session_id="my-run")]},
)
Every LLM call and tool result is recorded. Injections embedded in tool outputs are detected automatically.
Monitor AutoGen multi-agent conversations.
pip install autogen
from autogen import ConversableAgent
from agentforensics.integrations.autogen_tracer import patch_autogen
assistant = ConversableAgent(
name="assistant",
llm_config={"model": "gpt-4o-mini", "api_key": "sk-..."},
)
# Patch the agent - one line
tracer = patch_autogen(assistant)
print(f"Monitoring session: {tracer.session_id}")
# Use the agent normally
assistant.generate_reply(
messages=[{"role": "user", "content": "Analyse this webpage content: ..."}]
)
Each entry is one monitored conversation. The badge shows:
| Badge | Meaning |
|---|---|
CLEAN (green) |
No injection signals detected |
SUSPICIOUS (yellow) |
Low-confidence injection signals found |
COMPROMISED (red) |
High-confidence injection attack detected |
Click any session to open the full forensic report.
The report shows:
Open a session → click Download PDF in the top toolbar. The full report prints as a PDF with all evidence visible.
Click Live Monitor: ON in the top bar to watch injections arrive in real time via Server-Sent Events. Each alert shows the session, score, verdict, and evidence snippet.
AgentForensics groups similar injection attacks across sessions into campaigns - useful for identifying repeated or coordinated attacks.
Click Campaigns in the top navigation bar.
| Column | Description |
|---|---|
| Fingerprint ID | Unique ID for this injection pattern |
| Hits | How many times this pattern was detected |
| Sessions | How many sessions were affected |
| Campaign Name | Your label for this attack group |
| First Seen / Last Seen | Timeline of the attack |
| Snippet | Preview of the injection content |
Type a name in the Campaign Name field → click Save. Use this to track known attack groups (e.g. “Competitor scraping attack”, “Supply chain injection”).
Use the sort dropdown to order by:
If you see many similar fingerprints (e.g. variations of the same injection), click Deduplicate to merge them into one record.
If a session is incorrectly flagged, mark it as a false alarm to suppress future alerts for that content.
In the session list → click False Alarm below the session entry.
Click False Alarms in the top navigation bar to see all marked content.
In the False Alarms panel → click Unmark next to the entry. Future scans of that content will be evaluated normally again.
Tests against 7,560 real-world indirect injection payloads:
pip install datasets
python benchtest.py # first 2000 rows, heuristics only
python benchtest.py --ml # with ML classifier (slower)
python benchtest.py --all # full 7560 rows
python benchtest.py --verbose # show result per row
Tests each detection rule individually with hand-crafted samples. No internet required:
python benchmark.py
python benchmark.py --verbose # show score for every sample
python benchmark.py --ml # include ML stage
Results are saved to benchtest_results.json / benchmark_results.json.
Set these before launching the dashboard or running your app:
# Minimum score to flag an injection (default: 0.25)
export AF_INJECT_THRESHOLD=0.25
# Cosine similarity for grouping fingerprints (default: 0.70)
export AF_FP_THRESHOLD=0.70
# Disable ML stage for faster startup (default: false)
export AF_DISABLE_ML=true
# Custom model path (default: agentforensics/model/)
export AF_MODEL_PATH=/path/to/model
# Custom fingerprint database path
export AF_FP_DB=~/.agentforensics/fingerprints.db
# Sliding window size for multi-turn detection (default: 3)
export AF_WINDOW_SIZE=3
Windows (PowerShell):
$env:AF_DISABLE_ML = "true"
python -m agentforensics.cli dashboard
Error: address already in use
Another process is using port 7890. Either stop it or use a different port:
python -m agentforensics.cli dashboard --port 8080
The tool prints a warning and falls back to heuristics-only mode. Fix:
python scripts/download_model.py
Ctrl+Shift+P → Developer: Reload Windowexport AF_INJECT_THRESHOLD=0.15
The shortcut points to %LOCALAPPDATA%\AgentForensics\AgentForensics.exe. Re-run python install.py to recreate it, or launch directly from dist\AgentForensics.exe.
# Windows
netstat -ano | findstr :7890
taskkill /PID <PID> /F