Azure DevOps

Run a Console agent from an Azure DevOps Server (TFS) or Azure DevOps Services pipeline: generate release notes, review a change, triage test failures, gate a release on an agent's verdict. The pipeline creates a run, waits for it, turns the result into pipeline variables, and fails the step when the agent fails.

Why polling. Build agents usually sit behind a firewall and cannot receive a callback. The pattern here uses only outbound HTTPS: create the run, then long-poll it. Each poll holds the connection open on the server side until the run settles or the wait ends, so there is no busy loop and no inbound port.

The pattern

1. Create the run with POST /agents/{key}/runs and an Idempotency-Key made from the build id and step name. A retried step (or a re-run of the same stage) finds the same run instead of starting a second one.
2. Wait with GET /agents/runs/{id}?wait=50 in a loop, until the status is succeeded, failed or canceled.
3. Read output_parsed (a validated JSON object, when you sent a response_format) or output_text, and publish them with ##vso[task.setvariable …].
4. Fail the step on failed or canceled, with the error message from the run.
5. Cancel the run if the pipeline is cancelled. A final step with condition: always() cancels the run when it is still active, so a cancelled or timed-out pipeline does not leave an agent working (and spending).

Secrets

Keep the Console URL and the token in a variable group (Pipelines → Library) named console-agent: CONSOLE_URL as a plain variable and CONSOLE_TOKEN marked secret (the lock icon). Azure DevOps never passes secret variables to scripts automatically, so map them into the environment of each step explicitly, as below. Use a token of a dedicated service user whose project holds only the agents the pipeline may call.

Pipeline: release notes with structured output

# azure-pipelines.yml
trigger: none
pool: { vmImage: windows-latest }          # or your own agent pool

variables:
  - group: console-agent                    # CONSOLE_URL, CONSOLE_TOKEN (secret)

steps:
  - task: PowerShell@2
    name: agent
    displayName: Generate release notes with the agent
    env:
      CONSOLE_TOKEN: $(CONSOLE_TOKEN)       # secrets must be mapped explicitly
    inputs:
      targetType: inline
      pwsh: false
      script: |
        $ErrorActionPreference = 'Stop'
        [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12   # older Azure DevOps Server hosts

        $base    = "$env:CONSOLE_URL/api/client/v1"
        $headers = @{ Authorization = "Bearer $env:CONSOLE_TOKEN" }
        $key     = "$env:BUILD_BUILDID-release-notes"     # build id + step name

        $commits = git log --oneline "origin/main..HEAD" | Out-String
        $body = @{
          input = "Write release notes for these commits:`n$commits"
          metadata = @{ build = "$env:BUILD_BUILDNUMBER"; repo = "$env:BUILD_REPOSITORY_NAME" }
          response_format = @{
            type = 'json_schema'
            json_schema = @{
              name = 'release_notes'; strict = $true
              schema = @{
                type = 'object'; additionalProperties = $false
                required = @('version_bump', 'summary', 'breaking')
                properties = @{
                  version_bump = @{ type = 'string'; enum = @('major', 'minor', 'patch') }
                  summary      = @{ type = 'string' }
                  breaking     = @{ type = 'boolean' }
                }
              }
            }
          }
        } | ConvertTo-Json -Depth 12

        # 1. Create (async: returns at once). The idempotency key makes a retry of this step safe.
        $run = Invoke-RestMethod -Method Post -Uri "$base/agents/release-notes/runs" `
          -Headers ($headers + @{ 'Idempotency-Key' = $key }) -ContentType 'application/json' -Body $body
        Write-Host "Run $($run.id) is $($run.status)"
        Write-Host "##vso[task.setvariable variable=CONSOLE_RUN_ID]$($run.id)"     # for the cancel step

        # 2. Long-poll until the run settles (or 20 minutes pass).
        $deadline = (Get-Date).AddMinutes(20)
        while ($run.status -in 'scheduled', 'queued', 'running') {
          if ((Get-Date) -gt $deadline) { throw "Agent run $($run.id) did not finish in 20 minutes" }
          $run = Invoke-RestMethod -Uri "$base/agents/runs/$($run.id)?wait=50" -Headers $headers -TimeoutSec 70
        }

        # 3. requires_action means the agent waits for a person: not something a pipeline can answer.
        if ($run.status -eq 'requires_action') { throw "Agent run $($run.id) is waiting for approval" }

        # 4. Fail on failed / canceled
        if ($run.status -ne 'succeeded') {
          Write-Host "##vso[task.logissue type=error]Agent run $($run.id) $($run.status): $($run.error.message)"
          exit 1
        }

        # 5. Publish the result
        $notes = $run.output_parsed
        Write-Host "##vso[task.setvariable variable=VersionBump;isOutput=true]$($notes.version_bump)"
        Write-Host "##vso[task.setvariable variable=HasBreaking;isOutput=true]$($notes.breaking)"
        Set-Content -Path "$env:BUILD_ARTIFACTSTAGINGDIRECTORY/release-notes.md" -Value $notes.summary
        Write-Host "##vso[task.uploadsummary]$env:BUILD_ARTIFACTSTAGINGDIRECTORY/release-notes.md"

  # Always runs, also when the pipeline is cancelled or times out: stop the run if it is still active.
  - task: PowerShell@2
    displayName: Cancel the agent run if still active
    condition: always()
    env:
      CONSOLE_TOKEN: $(CONSOLE_TOKEN)
    inputs:
      targetType: inline
      script: |
        if (-not $env:CONSOLE_RUN_ID) { return }
        $base = "$env:CONSOLE_URL/api/client/v1"
        $h = @{ Authorization = "Bearer $env:CONSOLE_TOKEN" }
        try {
          $run = Invoke-RestMethod -Uri "$base/agents/runs/$env:CONSOLE_RUN_ID" -Headers $h
          if ($run.status -in 'scheduled', 'queued', 'running', 'requires_action') {
            Invoke-RestMethod -Method Post -Uri "$base/agents/runs/$env:CONSOLE_RUN_ID/cancel" -Headers $h | Out-Null
            Write-Host "Cancelled agent run $env:CONSOLE_RUN_ID"
          }
        } catch { Write-Host "Could not cancel run: $($_.Exception.Message)" }

A later step or stage reads the published values through the step's name: $(agent.VersionBump) in the same job, or dependencies.<job>.outputs['agent.VersionBump'] from another job. Cancelling a queued or scheduled run is immediate; a running one stops at its next checkpoint (about a second) and keeps its partial answer.

Bash and curl version (Linux agents)

steps:
  - bash: |
      set -euo pipefail
      BASE="$CONSOLE_URL/api/client/v1"
      AUTH="Authorization: Bearer $CONSOLE_TOKEN"

      RUN=$(curl -fsS -X POST "$BASE/agents/triage/runs" -H "$AUTH" -H "Content-Type: application/json" \
        -H "Idempotency-Key: ${BUILD_BUILDID}-triage" \
        -d "$(jq -n --arg t "$(cat test-failures.txt)" '{input: $t, class: "default", metadata: {build: env.BUILD_BUILDNUMBER}}')")
      RUN_ID=$(jq -r .id <<<"$RUN")
      echo "##vso[task.setvariable variable=CONSOLE_RUN_ID]$RUN_ID"

      STATUS=$(jq -r .status <<<"$RUN")
      while [[ "$STATUS" == scheduled || "$STATUS" == queued || "$STATUS" == running ]]; do
        RUN=$(curl -fsS "$BASE/agents/runs/$RUN_ID?wait=50" -H "$AUTH" --max-time 70)
        STATUS=$(jq -r .status <<<"$RUN")
      done

      if [[ "$STATUS" != succeeded ]]; then
        echo "##vso[task.logissue type=error]Agent run $RUN_ID $STATUS: $(jq -r '.error.message // empty' <<<"$RUN")"
        exit 1
      fi
      echo "##vso[task.setvariable variable=Verdict;isOutput=true]$(jq -r '.output_text' <<<"$RUN" | head -c 200 | tr -d '\n')"
    name: agent
    displayName: Triage test failures
    env:
      CONSOLE_TOKEN: $(CONSOLE_TOKEN)

  - bash: |
      [ -z "${CONSOLE_RUN_ID:-}" ] && exit 0
      curl -fsS -X POST "$CONSOLE_URL/api/client/v1/agents/runs/$CONSOLE_RUN_ID/cancel" \
        -H "Authorization: Bearer $CONSOLE_TOKEN" >/dev/null || true     # a finished run answers 409; that is fine
    displayName: Cancel the agent run if still active
    condition: always()
    env:
      CONSOLE_TOKEN: $(CONSOLE_TOKEN)

Or let Azure DevOps call Console

A pipeline step is the right tool when the build should wait for the agent and fail on its verdict. When you only want an agent to react to an event (a pull request is created, a build fails, a work item is filed) and nobody needs to wait, skip the pipeline: add a webhook trigger and point an Azure DevOps service hook at its URL. The service hook, not an agent in your pool, makes the call, so Console must be reachable from Azure DevOps (Services: from the internet; Server: from the network), the opposite of the polling pattern above.

	Pipeline step (this page)	Service hook + webhook trigger
Who calls whom	The build agent calls Console (outbound only)	Azure DevOps calls Console (inbound)
The result	Read in the step; the step can fail the build	Arrives later by callback, or comment from the agent's own tool
Setup	YAML and a token	A webhook trigger, a service hook, no token in Azure DevOps
Fits	Release notes, gates, test triage inside a build	PR review, build-failure triage, work-item triage

A complete pull request review recipe is on the webhook page.

Good practice

• Make the key specific. BUILD_BUILDID plus a step name means each build runs the step once, however often the step is retried. Add the stage attempt (SYSTEM_STAGEATTEMPT) if a manual re-run of a stage should start a new run.
• Put the question in input, the contract in response_format. A schema turns "free text to parse" into fields a pipeline can branch on. Invalid schemas fail at once with 400 invalid_response_format.
• Bound the wait. The loop above stops after 20 minutes. The server's own ceiling for a run is 30 minutes by default (AGENT_BACKGROUND_MAX_DURATION_MS), after which it fails with max_duration_exceeded.
• Tag runs with metadata (build number, repository). You can find them later with GET /agents/runs?metadata[build]=1234.
• Use "class": "deferred" for nightly jobs that may wait for capacity (run classes).
• Handle 429. agent_run_concurrency_limit means too many active runs for the tenant; retry with backoff, or use the deferred class. spend_limit_exceeded is not a retry case.
• Log the run id (Write-Host). It is the handle for Console's Sessions and Tracing pages when someone asks "why did the agent say that".

See also: GitHub Actions for the same pattern in YAML with curl and jq, and Use cases.