Use case

Agent Development With Tracing

Develop custom agents with Agent SDK and feed execution traces into Console so teams can compare runs, inspect tool behavior, and debug workflows faster.

• Stack Agent SDK + Console + Console SDK
• Pattern Agents · Observability
• Read ~11 min

Overview

Teams building custom agents usually hit the same wall: the agent works in a local test, but it is hard to compare runs, inspect tool sequences, and understand failures once multiple prompts, tools, and datasets are involved.

A practical pattern is to keep agent construction in Agent SDK while using Console as the place where traces are collected and reviewed.

When to reach for this use case

If your team needs the capabilities described above and you would rather build on proven primitives than wire one from scratch — this is the shape to start from.

Architecture

Agent SDK owns the runtime logic, tools, and control flow. Console SDK sends structured tracing data into Console, where product and platform teams can inspect sessions, thread-level workflows, latency, and token usage.

This creates a clean split: build agents where you need determinism, observe them where you need operational visibility.

1. Build The Agent In Agent SDK

Keep the runtime local to your application or service, but give each run a stable session and thread identity.

import { createSmartAgent, createTool } from '@cognipeer/agent-sdk';

const agent = createSmartAgent({
  name: 'PolicyReviewer',
  model,
  tools: [searchPolicyDocs, summarizeFindings],
  systemPrompt: 'Review uploaded policies and produce structured findings.',
  useTodoList: true,
  tracing: { enabled: true },
});

const sessionId = 'sess_' + Date.now();
const threadId = 'thread_policy-review-2026-03';

2. Ingest Execution Data Into Console

After a run completes, send the session summary and important events into Console for later comparison and debugging.

import { ConsoleClient } from '@cognipeer/console-sdk';

const client = new ConsoleClient({
  apiKey: process.env.COGNIPEER_API_KEY!,
  baseURL: 'https://console.example.com',
});

await client.tracing.ingest({
  sessionId,
  threadId,
  source: 'custom',
  status: 'success',
  startedAt,
  endedAt: new Date().toISOString(),
  durationMs: 1480,
  agent: {
    name: 'PolicyReviewer',
    version: '0.3.0',
    model: 'gpt-4o-mini',
  },
  summary: {
    totalInputTokens: 1240,
    totalOutputTokens: 420,
    totalCachedInputTokens: 0,
    totalBytesIn: 18000,
    totalBytesOut: 6200,
    eventCounts: {
      ai_call: 2,
      tool_call: 2,
    },
  },
  events: traceEvents,
  errors: [],
});

3. Compare Agent Iterations By Thread

Thread correlation is useful when you are iterating on the same workflow across prompt versions, model changes, or tool updates.

async function runReview(promptVersion: string, input: string) {
  const startedAt = new Date().toISOString();

  const result = await agent.invoke({
    messages: [{ role: 'user', content: input }],
  });

  await client.tracing.ingest({
    sessionId: sessionId + '-' + promptVersion,
    threadId,
    source: 'custom',
    status: 'success',
    startedAt,
    endedAt: new Date().toISOString(),
    durationMs: 1200,
    agent: { name: 'PolicyReviewer', version: promptVersion, model: 'gpt-4o-mini' },
    summary,
    events: buildTraceEvents(result),
    errors: [],
  });
}

Result

You get a development workflow that:

• Builds custom agents in Agent SDK without giving up observability
• Captures tool calls, latency, and token usage in Console
• Compares multiple agent versions through shared thread IDs
• Speeds up debugging for prompt, model, and tool-chain changes

---

Products used

This pattern combines 3 Cognipeer products.

@cognipeer/agent-sdk

Agent SDK

Deterministic TypeScript agent runtime — planning, tools, guardrails, sub-agents, and human-in-the-loop.

Cognipeer Console

Console

Self-hosted AI control plane — routing, embeddings, vectors, files, guardrails, and tracing.

@cognipeer/console-sdk

Console SDK

Typed TypeScript client for every Console endpoint, with streaming support.

See also

• @cognipeer/observability — ships traces from Agent SDK and other runtimes into Console for you, instead of assembling the ingest payload by hand.
• Tracing data model — the exact shape of the session, thread, summary, and event fields used in step 2.
• Manual ingestion — the Console-side reference for the tracing.ingest call this pattern makes directly.
• Console observability — how sessions, threads, and spend attribution are presented once the traces land.

Related use cases

Agents · Ops

SRE Incident Response Agent

Build an SRE agent that reads Confluence runbooks, ingests them into Console RAG, searches vector knowledge during incidents, and posts Jira updates through tools.

Agents · Planning

Autonomous AI Agents

Build agents that plan, execute tools, and deliver structured outputs with full deterministic control and observability.

Agents · Orchestration

Multi-Agent Orchestration

Compose multiple specialised agents that hand off tasks, share context, and collaborate on complex workflows.

See every pattern on the use case index, or browse all Cognipeer libraries.