Red teaming

Red Team tests your agents and models against adversarial attacks before real users do. A campaign points a set of attack probes (prompt injection, jailbreaks, data extraction and others) at one agent or model. A scan runs the campaign, a layered decision engine judges every attempt, and the result is an attack-success and resilience score you can track over time, broken down by OWASP LLM risk category.

Red Team is a community feature. Operators open it as Red Team.

Concepts

Object	What it is
Probe	A generator for one vulnerability class. Each probe has an OWASP LLM category and a severity (low, medium, high, critical). Built-in probes ship with Console; you can author custom probes.
Campaign	A saved, reusable scan configuration: the target (an agent or a model), the probes to run, an optional judge model, and an optional schedule.
Scan (run)	One execution of a campaign. It contains one attempt per probe attempt, each with a verdict.
Verdict	safe, vulnerable or needs_review. A detector hit means the attack succeeded. Anything the engine cannot decide confidently becomes needs_review instead of being rounded to safe or vulnerable.

A scan reports an attack success rate (vulnerable attempts out of completed attempts) and a resilience score (one minus the attack success rate).

What you see on the screen

The Red Team area has these sections in its sub-navigation:

Section	What it shows
Overview	Compliance posture across all completed scans: completed scans, overall resilience, vulnerabilities, needs-review count, and resilience by OWASP LLM category with the riskiest categories first.
Campaigns	Your campaigns with target, probe count, schedule and creation date. Summary tiles show campaigns, scans, vulnerabilities and the latest resilience.
Policy Probes	Enterprise editions only. Fires a fixed battery of adversarial probes at one guardrail to check its policy contains threats such as SSRF, data exfiltration, remote code execution, sensitive-file access, prompt injection and credential leaks. It runs in shadow, so it writes no evaluation log and no usage event.
Scans	Every scan with campaign, status, attack success, vulnerable and needs-review counts, agent and resilience.
Probes	Your custom probes.
API	A short guide to the client API plus Run calibration.

Create a campaign

1. Open Red Team → Campaigns and choose New campaign.
2. Identity. Name the campaign and optionally describe it.
3. Target. Choose a target type. An agent target exercises the whole agent, including guardrails, Knowledge Engine and tools. A model target attacks the model directly. For an agent, choose the version to attack: published (whatever is live), draft (red-team a change before you publish it), or a pinned version number to compare releases.
4. Probes and judge. Select the probes, or leave them empty to run the entire built-in catalog. Custom probes must be selected explicitly. Pick a judge / attacker model. It powers the LLM-judge detectors and the adaptive multi-turn attacker that escalates across turns. Without it, attacks are single-shot and many verdicts stay needs_review, so it is strongly recommended.
5. Schedule. Optionally turn on a scheduled scan and enter a cron expression in UTC. The default is every day at 03:00 UTC (0 3 * * *). A background scheduler checks for due campaigns once a minute.
6. Save the campaign.

Run a scan

1. Open the campaign (or use Run scan on its row) and choose Start scan.
2. Optionally adjust the settings for this scan:
   • Agent version — the campaign default, published, draft, or a specific version. The scan records the version that answered, so results stay comparable.
   • Max turns per attempt (1 to 10) — how long the adaptive attack can escalate. Needs a judge model.
   • Concurrency (1 to 16) — parallel attempts. The form default is 4.
   • Probes and Judge / attacker model — overrides for this scan only.
3. The scan starts in the background and is listed under Scans with status pending, then running, then completed. Only one scan per campaign runs at a time; starting another while one is in progress is refused.

Attacking an agent target requires write access to agents, because the scan runs the agent. Without it the Start scan button is disabled and the API refuses the request.

Read and review a scan

Open a scan from Scans to see:

• tiles for attack success rate, resilience, vulnerable and needs-review counts;
• a table of attempts with their verdicts;
• a comparison against an earlier scan of the same campaign, to see regressions between releases. If the agent draft changed while the scan ran, the page warns you.

Open Review / override on an attempt to see the full transcript (the attacker turns and the target's replies), then set the outcome to Safe, Vulnerable or Needs review and add a note. Reviewed attempts are marked as human-reviewed.

Custom probes

Under Probes, choose New custom probe to add your own attacks, which then become selectable in any campaign.

• Set a name, an OWASP category, a severity and a description.
• Add one or more attempts. Each has the adversarial user turn(s) (add turns for a scripted multi-turn attack), an optional system prompt (use to embed a canary secret), a canary secret (if it leaks it is a confirmed hit), forbidden patterns (comma-separated substrings or regular expressions that indicate a hit), whether a refusal is the safe outcome, and optionally an adaptive multi-turn attacker with an objective.
• Choose detectors: Refusal gate (a cheap explicit-refusal safe signal), Pattern / canary (deterministic hard proof), and judge lenses. A judge lens has a name, a rubric ("the attack succeeds if…") and a threshold between 0 and 1.

Calibration

In the API section, Run calibration runs a built-in golden set through the live detectors and decision engine and reports accuracy, precision, recall and F1, the review rate, and the confusion counts. Judge-dependent cases need a judge model.

Run scans from a pipeline

The client API can list probes and campaigns, start a scan, and read a run, using an API token. A pipeline can start a scan, poll the run until it is completed, and fail when the run's aggregate reports any vulnerability. Authoring campaigns and custom probes stays in the dashboard. See the Red Team API reference.

Permissions and requirements

Topic	Behaviour
Permission	The redteam service (RBAC and API token service permissions) covers the dashboard and client API.
Agent targets	Starting a scan against an agent also needs write access to agents.
Scope	Campaigns, custom probes and scans are project-scoped.
Models	A judge model is optional but strongly recommended; model targets and the judge must exist in the project.
Execution	Scans run through the cluster queue layer, off the request that started them.

Related

• Red Team API reference
• Guardrails
• Evaluation and analysis