Nothing works until rows arrive.
TokenPlane does not talk to any vendor API and has no agent of its own. It reads one ClickHouse database that an OpenTelemetry collector writes to. This page is what the dashboard's first-run banner points at: what the pipeline expects, how to make each supported client emit it, and how to tell which stage is stuck when it doesn't.
Anything that speaks OTLP can feed it.
your client
│ OTLP/HTTP :4318 (http/protobuf)
▼
otel-collector
│ native :9000
▼
ClickHouse · otel
├─ raw otel_logs / otel_metrics_* / otel_traces
└─ typed otel_events_api_request ← every dollar
| Endpoint | http://localhost:4318 (OTLP/HTTP). gRPC on :4317 also works |
|---|---|
| Protocol | http/protobuf |
| Signals | logs and metrics at minimum. Traces are used only by the gateway path below |
| Database | otel, created by the collector (create_schema: true) plus queries/apply_schema.py for the typed tables and views |
The one table the dashboard cannot do without is otel_events_api_request —
one row per API round-trip, and the denominator of every cost figure on every page. It is
populated by a materialized view over the raw log stream, so a client that exports
metrics but no logs will show session counts and little else.
Native OpenTelemetry, no proxy.
Claude Code emits OpenTelemetry itself. Copy the env block from
claude-code/settings.json into your ~/.claude/settings.json.
The load-bearing variables:
| Variable | What it does |
|---|---|
CLAUDE_CODE_ENABLE_TELEMETRY=1 | master switch |
OTEL_METRICS_EXPORTER=otlpOTEL_LOGS_EXPORTER=otlp | export both signals |
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobufOTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 | point at the collector |
OTEL_LOG_USER_PROMPTS=1OTEL_LOG_TOOL_DETAILS=1 | enrich log events with content |
OTEL_LOG_RAW_API_BODIES=file:<repo>/data/raw-api-bodies | write full request/response bodies to disk — this powers the Session Metrics context-window breakdown (inline mode truncates at 60 KB and the JSON doesn't even parse) |
OTEL_METRICS_INCLUDE_SESSION_ID=true (and the other OTEL_METRICS_INCLUDE_*) | put session.id on metrics rows so they join to sessions |
OTEL_LOG_USER_PROMPTS records
your prompt text into ClickHouse, and OTEL_LOG_RAW_API_BODIES writes complete
request/response JSON (prompts, completions, file contents in context) to unencrypted files
on disk. Every row also carries your account email. If the store will hold anyone else's
sessions, use the partner defaults in deploy/SETUP.md instead — both content
flags off — and get an explicit OK before enabling them.
Gotchas — read these before concluding capture is broken
- Env vars are read at process startup. The session in which you edit
settings.jsonwill never see its own telemetry. Relaunch, then check for rows. - The desktop app buffers event-logs until a clean exit. A running
desktop session exports metrics on a ~60 s timer (so session counts, cost
counters and lines-of-code appear live), but its per-event log rows — everything Session
Metrics charts per turn — land only when the app quits gracefully. A force-quit loses the
buffer. The CLI (
claudein a terminal) streams event-logs within seconds, so use the CLI if you want to watch a session live. Corollary: a session cannot watch its own telemetry — desktop flushes on exit, and even CLI rows trail by ~20 s. - Don't add the beta-tracing flags (
OTEL_TRACES_EXPORTER,CLAUDE_CODE_ENHANCED_TELEMETRY_BETA, …). Empirically they suppress live log export entirely — sessions produce zero event rows until shutdown. - Raw bodies grow fast — a heavy day writes gigabytes, since every API
round-trip late in a big session rewrites a ~1 MB conversation JSON. A
SessionStarthook (.claude/hooks/body-retention.py) deletes bodies older than 14 days; the dashboard tolerates missing files.
Native OpenTelemetry too — three pieces, in order.
Codex CLI emits OpenTelemetry itself, under a ChatGPT-plan sign-in — no API key and no gateway. It needs three pieces of client-side setup where Claude Code needs one, and the failure modes are quieter, so do them in order and confirm after each.
1. The [otel] block
Merge codex/config.toml into your ~/.codex/config.toml — merge,
don't overwrite; the file also holds your project trust levels. The load-bearing lines:
| Setting | What it does |
|---|---|
[otel.exporter.otlp-http](and /v1/traces, /v1/metrics for the other two exporters) | point each signal at the collector — the full signal URL, one per exporter. Codex does not append the path: given a base URL it POSTs to /, the collector answers 404, and the CLI says nothing while the completion succeeds. The opposite of Claude Code's single base-URL variable |
protocol = "binary" | http/protobuf |
log_user_prompt = true | put the prompt on the codex.user_prompt log row. Required for explicit skill activations: a $skill mention exists nowhere else, so the collector reads the mention from the prompt and then deletes the prompt before it reaches ClickHouse. The text transits the local collector and is never stored |
environment = "…" | a fleet label on every row; not a session key |
Traces are not optional here: the Codex cost row is built from the turn span, so a config that exports logs and metrics but not traces produces sessions with identity and no spend.
2. The ticket hook
Copy codex/hooks.json to ~/.codex/hooks.json and fill in the
absolute path to codex/hooks/ticket-capture.py. It is the same
<TICKET-KEY>-… branch convention as Claude Code and writes the same
ticket rows. Then establish hook trust — Codex will not run a hook it has
not been told to trust, and it does not warn when it skips one:
- interactive
codex, once per machine: accept the directory-trust prompt, then — at the “Hooks need review” prompt, whose default isReview hooks— choose2. Trust all and continue. Codex records a content hash of the hook entry under[hooks.state]inconfig.toml; from then on unattendedcodex execruns the hook as well (measured on 0.149.0); - unattended
codex execon a machine where nobody has done that once: pass--dangerously-bypass-hook-trust, or the session lands untagged and shows under Outcomes' (untracked) bucket.
Measured before this was documented: the hook fired on 31% of Codex sessions against 100% on Claude Code, entirely from unattended runs. Diagnostics → Capture health shows the ratio per client so you can see whether it is firing.
3. Project trust
~/.codex/config.toml needs [projects."/path/to/repo"] trust_level =
"trusted" for the working tree you run in, or Codex runs read-only and the hook
cannot read the branch.
What to expect once it works
A turn appears about a minute after it finishes, not in seconds: the cost
row is joined from four spans by a refreshable view on a 60 s timer. The per-session
websocket warm-up request is real traffic that turn accounting excludes; Session Metrics
states it above the charts rather than hiding it. A sub-agent Codex spawns is its own
thread and shows as agent spend under the parent session and the turn that spawned
it — it is not a separate session — and it waits for the parent turn to close
before it files. A $skill mention shows on the Skills tab once Codex's own
skill-injected counter confirms it ran (on Codex 0.149.0 that counter covers host-catalogue
skills only; plugin-skill mentions appear from 0.151.0). A skill the model read on its own
shows as a proactive activation, excluded from the tab's user-invoked figures like Claude
Code's.
arguments and
output and the prompt before ClickHouse, keeping only the keys it extracted (a
skill name, a child thread id, a mention). Set
COLLECTOR_RETAIN_CODEX_TOOL_CONTENT=1 on the collector to keep all three — not
on a store that holds anyone else's sessions.
Gotchas — Codex
- Silence is the failure mode. A wrong endpoint, an untrusted hook, and a
missing exporter all succeed from the CLI's point of view.
RUST_LOG=debug codex exec …is the only way to see an export fail; for the hook, look at Capture health's ticket ratio. - Three server-side behaviours changed inside one week on a pinned client — the warm-up size, the model catalogue, and the quota-window shape. Rate-card and warm-up figures are derived from the stream, never hard-coded, but expect the numbers to move without an upgrade.
- Fast mode is invisible. Codex does not report the service tier, and a fast turn prices at 2–2.5× standard, so a derived-cost figure can be low by half on fast turns. Capture health bounds the exposure; there is no fix on the client side today.
Everything else goes through the gateway.
Clients that don't emit OpenTelemetry themselves are routed through the LiteLLM gateway
(compose service litellm), whose OTel callback exports spans to the same
collector. docs/capture-coverage.md carries the per-route maturity table and
exactly which fields survive each hop.
client
│
LiteLLM gateway
│ OTLP (OTel callback)
▼
otel-collector ──▶ otel_traces
│ queries/ingest_litellm.py
▼ batch — you run it
otel_logs + otel_events_api_request
This path is proven, not turnkey, and the honest limits are:
- The mapper is a batch job, not a standing ingest. Gateway spans land
in
otel_traceson their own, but nothing maps them into the analytic tables until you runpython3 queries/ingest_litellm.py. Run--inspectfirst — each provider's usage and cache fields are their own dialect, and the mapper pins keys against what a real span actually carried. - The gateway has no session or turn concept.
session.id/prompt_idcome from request metadata the caller tags, so grouping by session only works if the caller sets it. - Cost trustworthiness varies by provider. For
claude-*model ids LiteLLM's own cost map is close to direct (~1.5%). For routes where it isn't — hashed model names, zeroed cache cost — the mapper re-derives cost from tokens with that provider's cache pricing rather than believing the span. - The ticket-attribution hook does not fire for gateway traffic. It is
client-side, so gateway callers land in the Outcomes page's
(unticketed)bucket unless the caller tags the metadata itself.
Did rows actually land?
Fastest first:
# 1. Is the collector listening? The OTLP receiver is POST-only on /v1/*, so a # GET to a real OTLP path answers 405 — any HTTP status proves something is # there. Only "000" means nothing answered. (Don't test / with curl -f: the # receiver 404s that path even when healthy, so -f reports a false failure.) $ code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:4318/v1/logs) $ [ "$code" = "000" ] && echo "collector NOT listening" || echo "listening (HTTP $code)" # 2. Has anything been written at all? $ curl -s "http://localhost:8123/?user=otel&password=$CH_PASSWORD" \ --data "SELECT count() FROM otel.otel_logs" # 3. The one that matters — the table every dollar comes from: $ curl -s "http://localhost:8123/?user=otel&password=$CH_PASSWORD" \ --data "SELECT count() FROM otel.otel_events_api_request" # 4. Codex specifically — spans arrived, and the refresher turned them into cost # rows (allow a minute; the second count lags the first by one refresh): $ curl -s "http://localhost:8123/?user=otel&password=$CH_PASSWORD" \ --data "SELECT countIf(SpanName = 'session_task.turn') FROM otel.otel_traces WHERE ServiceName = 'codex_exec'" $ curl -s "http://localhost:8123/?user=otel&password=$CH_PASSWORD" \ --data "SELECT count() FROM otel.otel_events_api_request WHERE query_source LIKE 'codex%' OR query_source = 'agent:builtin:codex-subagent'"
In the dashboard, Diagnostics answers the same question with no shell: the liveness panel shows the age of the newest part per table, so a stalled collector reads as a growing age rather than an absence you have to interpret.
One command, one file, one FAIL line.
Run the bundled diagnostic from the install folder. It walks every stage — engine, collector, ClickHouse, dashboard, client settings — and prints one OK/FAIL line each, so the break point is the first FAIL rather than a guess:
$ ./diagnose.sh # writes ./tokenplane-diag.txt
On Windows, run it from the WSL2 shell the same way, or use the native PowerShell version:
powershell -ExecutionPolicy Bypass -File .\diagnose.ps1. The report is
redacted by default — no passwords, tokens, or telemetry content, just
status and counts — so it is safe to send on. (--full adds operational log and
error detail, still with secrets and prompt content stripped.)
| Symptom | Usual cause |
|---|---|
receiving / stored WARN, everything else OK | Nothing has been sent yet. Check the client is actually configured and was restarted after the change |
otel_logs at 0 mid-session, metrics flowing | Normal for a Claude Code desktop session — event-logs flush on clean exit |
Rows in otel_logs but otel_events_api_request empty | The typed tables or their materialized views are missing. Run python3 queries/apply_schema.py |
Gateway traffic in otel_traces only | Expected — run queries/ingest_litellm.py (§4); the mapper is a batch step |
Codex session in otel_logs, nothing in otel_traces | The trace exporter is missing or points at a base URL. Every Codex exporter needs its full signal URL (§3) |
Codex spans in otel_traces, no cost rows after a minute | The refreshable promotion is stopped or absent. Diagnostics → Codex promotion; then python3 queries/apply_schema.py, and --promote-codex to replay a gap |
| Codex spend all under (untracked) on Outcomes | The ticket hook is not trusted — the “Hooks need review” prompt was answered with Review or Continue without trusting, or nobody has run interactive codex on that machine yet (§3). Capture health shows the per-client ratio |
A $skill mention never reaches the Skills tab | log_user_prompt is false, or the skill is served by a plugin on Codex < 0.151 (§3). Capture health's Codex tripwire compares the client's counter to the promoted rows |
| Collector reachable, ClickHouse refusing | Credential mismatch. CH_USER / CH_PASSWORD must agree between .env, the collector, and the backend |