How agents call models
An agent does not talk to a provider on its own. Every model call it makes is a call into the Console gateway, in process, with the same pipeline a direct /chat/completions request goes through.
Available from the next Console release
Agents on the gateway pipeline arrive with the next Console release. Before, an agent built its own provider client, and Model Hub settings such as guardrails or replicas did not apply to its calls.
What counts as a model call
All of these use the gateway:
- the agent's main model and any additional model a caller picked,
- the models of sub-agents the agent delegates to,
- the call that generates a conversation title,
- the model check when you save or test an agent,
- calls routed through a Dynamic LLM.
What you configure once, and where it applies
Whatever is set on the Model Hub record applies to an agent call exactly as it applies to a direct call:
| Setting | Effect on an agent call |
|---|---|
| Guardrails on the model | Input guardrails judge the user's message once per run (not again on every step of a tool loop). A block ends the call with the usual 400 guardrail_block. Output guardrails apply to each answer |
| Semantic cache | Bypassed for agent and tool calls, so an agent never gets a stale answer to a tool turn |
| Replica pool | A call fails over to another replica before the first byte, as for any caller |
| Dynamic LLM routing | The router picks the concrete model; the call is logged on the model that answered |
| Parameter rules | The model's request rules (unsupported parameters stripped, forced values) apply |
| Reasoning | The model's default effort and Return reasoning to callers apply. The agent's own pick for the model outranks the default; Show reasoning on the agent hides the text |
| Upstream API | An agent on a model set to Responses calls Responses; nothing changes for the agent's clients |
| Pricing and spend | Priced per call on the model that answered; budgets settle from the same usage |
Usage and logs: one row per call
Each model call writes one usage row, on the agent route, tagged with:
- the agent key and the session,
- the user the run is for,
- the purpose of the call (for example the answer, a title, a sub-agent step).
So tracing, the Model Hub logs and usage reports separate the calls of one agent by model and by purpose, and a title never looks like an answer. The prompt can stay out of the rows when you prefer not to store it.
The row's outcome follows what happened:
| Outcome | When |
|---|---|
success | The call finished |
cancelled | The run was stopped (a user pressing stop, a cancelled run or a closed connection) while the call was in flight. The tokens generated until then are still recorded and billed. Cancelling also stops the upstream request, so a model that is silently thinking stops too |
error | The call failed. The row is written for callers that ask for it |
A cancelled call is not an error and does not count toward the error rate. See Model Hub logs.
What this means in practice
- Put a guardrail on a model and every agent using it is covered, without touching the agent.
- Move a model behind a replica pool or a Dynamic LLM and agents follow.
- Change a model's reasoning default and agents that do not pick their own level follow.
- Cost and token reports for the model include the agent traffic, and you can still split it by agent.