AI coding-agent telemetry in CI (GitHub Actions)

Run Claude Code, Codex CLI, or another OTLP-capable coding agent inside a GitHub Actions job and capture its telemetry under the same managed redaction/forwarding policy you use on developer workstations.

This is the CI counterpart to workstation enrollment: instead of a launchd / systemd service, the supervisor runs as a short-lived background process on the runner itself, for the lifetime of one job.

Where the supervisor runs

┌ GitHub Actions runner (one job) ───────────────────────────────┐
│                                                                 │
│   claude / codex  ──OTLP──▶  127.0.0.1:4319                      │
│                                   │                             │
│                        ampbase supervisor (background)          │
│                        • loopback OTLP gateway                   │
│                        • events pipeline: normalize → redact    │
│                          → retain → forward                     │
│                                   │                             │
└───────────────────────────────────┼─────────────────────────────┘
                                    │
                 ┌──────────────────┴──────────────────┐
                 │ OpAMP: policy in                    │ events out
                 ▼                                     ▼
             Ampbase                            Ampbase ingest
        (config and policy —                    and/or your SIEM
         never telemetry)

The supervisor runs on the runner, not on a separate host and not as a service. This is deliberate:

• Policy stays in the loop. The runner enrolls into a dedicated ci-runners Channel and pulls its redaction/forwarding policy over OpAMP — the same versioned, canaried policy machinery as any other fleet. A CI run is not a special case; it is just another workstation segment.
• Nothing raw leaves the runner. The agent exports to loopback only; the gateway redacts by policy before anything is forwarded. There is no auth header on the loopback hop — the supervisor attaches the tenant bearer when it forwards upstream.
• Ephemeral by construction. The gateway, its JSONL retention, and the supervisor process all live in a mktemp dir that vanishes with the job.
• Early events fail safe. The supervisor starts under the local default policy — metadata-only, both forwarding gates off — before the OpAMP policy for the ci-runners Channel lands. Any events emitted in that startup window (a real concern for short jobs) are redacted at metadata-only and are not forwarded; forwarding is a no-op until an admin-enabled policy and its destination arrive over OpAMP. A CI job can never leak more than the local default before the managed policy takes effect.

If you are running a hosted cloud agent that cannot host a sidecar (e.g. Claude Cowork), use the direct org-scoped OTLP ingest route instead — see cloud coding-agent OTLP intake. That path trades the in-loop redaction for zero-infrastructure setup.

Setup

1. Create a ci-runners Channel (kind: AI coding agents) and mint a channel-scoped agent key. One key is shared by all your CI runners, exactly as a server fleet shares one — per-run identity comes from the supervisor's instance UID, not the key.
2. Store the key as a GitHub Actions secret, e.g. AMPBASE_CI_AGENT_KEY.
3. Make the ampbase binary available on the runner — either install it in a prior step or pass a pinned download-url to the action.

Usage

Composite action

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: ./.github/actions/coding-agent-telemetry
        with:
          agent-key: ${{ secrets.AMPBASE_CI_AGENT_KEY }}
          opamp-endpoint: https://acme.ampbase.io/v1/opamp
          # A binary you trust; no default URL is baked in.
          download-url: https://ampbase.io/releases/v0.6.0/ampbase-x86_64-unknown-linux-gnu
          command: claude -p "Review the changes on this branch for regressions"
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

The action starts the gateway, waits for it, runs command with the OTLP env vars pointed at loopback, then flushes and tears the supervisor down. The job's exit code is the agent command's exit code.

Wrapper script (any CI)

The action is a thin wrapper over run-with-telemetry.sh, which is CI-agnostic:

AMPBASE_BIN=/usr/local/bin/ampbase \
AGENT_KEY="$AMPBASE_CI_AGENT_KEY" \
OPAMP_ENDPOINT=https://acme.ampbase.io/v1/opamp \
.github/actions/coding-agent-telemetry/run-with-telemetry.sh \
  -- claude -p "Review the changes on this branch"

What the agent needs

The wrapper sets the coding agent's OTLP export env vars for you:

The low export interval matters: CI jobs are short, and the OTLP SDK's default 60-second batch window would drop the final (often only) batch when the agent process exits. The wrapper also sleeps briefly after the agent exits (FLUSH_SECONDS, default 3s) so the last window flushes before the gateway is stopped.

Runtime coverage notes

• Claude Code exports metrics + logs from these env vars directly.
• Codex CLI honours the standard OTEL_EXPORTER_OTLP_* pair, but note that as of v0.117.0 codex exec (the non-interactive form you'd use in CI) emits traces and logs but not metrics, and codex mcp-server emits no OTel at all (openai/codex#12913). Logs/events still flow.