Run classes

Every agent run belongs to one of two classes. The class decides which queue it waits in, how many runs of that class may be active, and how urgent it is. The same two classes exist on Console SaaS and on-prem installs.

	default	deferred
Meant for	Interactive and near-real-time work: chat, an API call someone waits for, a CI step	Batch-like work that can wait: nightly reports, backfills, bulk document analysis
How to ask	Nothing (the default)	"class": "deferred" on the runs API, service_tier: "flex" on the OpenAI endpoints, or class on a trigger
Execution	Inline (held connection) or queued	Always queued, started when a slot frees
Cap	Active runs per tenant (default 250), counting inline, queued and running	Running at once per tenant (default 2); queued per tenant (default 10 000)
When over the cap	429 agent_run_concurrency_limit at once	Queued up to the queued cap, then 429
Waiting	Short	Up to AGENT_DEFERRED_MAX_WAIT_MS (24 h), then failed / expired
Workers	AGENT_RUN_CONCURRENCY per node	AGENT_DEFERRED_CONCURRENCY per node (half of the default workers, at least 1)

Deferred runs never take interactive capacity: they have their own queue and their own workers, so a thousand queued nightly jobs cannot slow a chat down.

Asking for a class

# Runs API
curl -X POST "$CONSOLE_URL/api/client/v1/agents/analyst/runs" \
  -H "Authorization: Bearer $COGNIPEER_API_TOKEN" -H "Content-Type: application/json" \
  -d '{ "input": "Analyse contract 17", "class": "deferred" }'

# OpenAI Responses
curl -X POST "$CONSOLE_URL/api/client/v1/responses" \
  -H "Authorization: Bearer $COGNIPEER_API_TOKEN" -H "Content-Type: application/json" \
  -d '{ "model": "agent:analyst", "input": "Analyse contract 17", "service_tier": "flex", "background": true }'

A deferred run is never answered inline. A request that waits (wait on the runs API, or an OpenAI call without background) polls the run for you up to the wait window and then answers 202, exactly as for any slow run. For deferred work, do not wait: take the run id and collect the result later through a callback or webhook, or by polling.

run_at and delay_seconds combine with a class: the run is scheduled, becomes due, and then waits for a slot like any queued run. The 24-hour limit counts from when the run became due, not from when you created it.

A trigger sets the class of every run it creates. A webhook trigger fed by a busy sender (every push, every pull request update) belongs in deferred, so a burst of events queues instead of using interactive capacity; the sender gets its 202 as soon as the run is queued either way.

Lifecycle

scheduled ──(due)──► queued ──(slot)──► running ──► succeeded | failed | canceled
                       │                   └──► requires_action (waits for a person)
                       └──(waited too long)──► failed, error.type "expired"

Status	Meaning
scheduled	Has a start time in the future (scheduled_for). Counts for neither cap on the default class and holds no session lock.
queued	Waiting for a worker (default) or a deferred slot.
running	Executing.
requires_action	Paused for an approval or a question. A person must answer; see approvals.
succeeded, failed, canceled	Terminal. failed carries error.type: agent_error, max_duration_exceeded, worker_lost, expired, interrupted, …

A deferred or scheduled run that is not started within AGENT_DEFERRED_MAX_WAIT_MS after it became due fails with expired. A run that was running when the server shut down and could not finish within the drain time ends failed with interrupted, and is not retried automatically (it may already have had side effects): start it again yourself. Queued runs survive a restart and are published again at boot.

See the queue

curl "$CONSOLE_URL/api/client/v1/agents/runs/queue" -H "Authorization: Bearer $COGNIPEER_API_TOKEN" | jq

{
  "object": "agent.run_queue",
  "default":  { "running": 3, "queued": 0, "scheduled": 1, "limit": 250 },
  "deferred": { "running": 2, "queued": 148, "scheduled": 12, "running_limit": 2, "queued_limit": 10000, "oldest_queued_at": 1760000000 },
  "project": { "default": { "running": 1, "queued": 0, "scheduled": 0 }, "deferred": { "running": 0, "queued": 20, "scheduled": 0 } }
}

The counts are tenant-wide; project repeats them for the token's project. Use it to decide whether to submit more, or to alert on a growing deferred backlog (oldest_queued_at). To list the waiting runs themselves: GET /agents/runs?status=queued,scheduled&class=deferred.

429 semantics

Code	Class	Meaning	What to do
agent_run_concurrency_limit (scope: tenant or project, class: default)	default	The tenant (or project) already has the maximum number of active default-class runs. Counts inline, queued and running runs.	Retry with backoff, or send the work as deferred.
agent_run_concurrency_limit (class: deferred)	deferred	The tenant already has AGENT_DEFERRED_MAX_QUEUED_PER_TENANT queued or scheduled deferred runs.	Slow the producer; watch the queue endpoint.
agent_quota_exceeded	any	The agent-run quota is used up (Retry-After: 60).	Wait, or raise the quota. A run counts a quota unit only once it is admitted: a call refused for a conflict or a cap costs none.
spend_limit_exceeded	any	The tenant's spend limit is used up.	Not a retry case; raise the limit or wait for the period to reset.

The error body names the limit and scope; the response is refused before a run is created, so a retry never duplicates work. A resume of a paused run (after an approval) is not refused by the cap: it continues work that was already accepted. agent_run_concurrency_limit carries Retry-After: 30. A stream is checked against the caps and the quota before it opens, so over a limit you get this plain JSON 429, not an error event; and the session that the refused call created is removed again.

A scheduled run is promoted to queued only while the tenant (or project) is under its default-class cap: at the cap it stays scheduled and starts when a slot frees up, so it can start later than its run_at.

Settings

Variable	Default	Meaning
AGENT_RUN_MAX_ACTIVE_PER_TENANT	250	Default class: queued + running runs of all kinds per tenant; 0 = no cap. The effective cap is the smaller of this and the tenant quota maxConcurrentAgentRuns.
AGENT_RUN_MAX_ACTIVE_PER_PROJECT	0	The same per project; 0 = off.
AGENT_RUN_CONCURRENCY	4	Default-class queue workers per node.
AGENT_DEFERRED_MAX_RUNNING_PER_TENANT	2	Deferred runs running at once, per tenant.
AGENT_DEFERRED_MAX_QUEUED_PER_TENANT	10000	Deferred runs queued or scheduled, per tenant.
AGENT_DEFERRED_CONCURRENCY	half of AGENT_RUN_CONCURRENCY, at least 1	Deferred queue workers per node.
AGENT_DEFERRED_MAX_WAIT_MS	86400000 (24 h)	A deferred or scheduled run not started this long after it became due fails with expired.
AGENT_BACKGROUND_MAX_DURATION_MS	1800000 (30 min)	Wall-clock ceiling of any run.
AGENT_SYNC_TIMEOUT_MS	600000 (10 min)	The wait window of a synchronous call (not a limit on the run).

The full list, with the callback, drain and trigger settings, is in Configuration.

Sizing guidance

• A run is mostly waiting on a model. One node with AGENT_RUN_CONCURRENCY=4 is four turns started at once from the queue; inline calls (answered on the request that carried them) are bounded by the tenant cap, not by the workers. Raise the number of nodes, not just the concurrency, when CPU or memory is the limit.
• Interactive tenants: keep the default cap generous (250 is a ceiling, not a target), and set AGENT_RUN_MAX_ACTIVE_PER_PROJECT if one project should not starve another. The effective cap never exceeds the tenant's own quota.
• Batch tenants: leave AGENT_DEFERRED_MAX_RUNNING_PER_TENANT at 2 until you have measured. Each running deferred run holds a model call; the model provider's rate limit is usually the real ceiling. Raise it in small steps and watch 429s from the provider.
• SaaS: the limits are per tenant, so one customer's backlog never touches another's. On-prem: you are the only tenant (or a few); the same settings protect your model providers and your database.
• Time budgets: set the window (AGENT_SYNC_TIMEOUT_MS) below the shortest timeout of your load balancer or gateway, so a slow call becomes a clean 202 instead of a proxy error; the run keeps going either way.
• Shutdown: on SIGTERM a node waits AGENT_RUN_DRAIN_TIMEOUT_MS (10 s by default) for its local runs, then marks the rest interrupted, before the HTTP server starts waiting for its open connections (a caller waiting on such a run gets 503 agent_run_interrupted, and open streams end). Give the pod a termination grace period larger than that, and make orchestrators drain nodes before stopping them.