OTel Data Flow — VS Code Copilot Chat Extension

Complete tracing architecture for foreground agent, CLI background agent, CLI terminal, and Claude Code

Provider A (Extension NodeOTelService)
Provider B (SDK BasicTracerProvider)
Bridge SpanProcessor
Debug Panel Consumer
External Exporter
Synthetic Spans (ext-created)
Overview (All Agents)
Foreground Agent
CLI Background
CLI Terminal
Claude Code
VS Code Extension Host Process Provider A: NodeOTelService Registered at extension activation tracer A (stored ref, survives override) service.name = "copilot-chat" Emitters: onDidCompleteSpan, onDidEmitSpanEvent Provider B: SDK BasicTracerProvider Created by OtelLifecycle.initializeOtelSdk() Replaces Provider A as global service.name = "github-copilot" SpanProcessors: BatchSP + BridgeSP CopilotCliBridgeSpanProcessor Added to Provider B after trackSession() Maps traceId → sessionId, injects CHAT_SESSION_ID Filters: only forwards registered traceIds Foreground Agent toolCallingLoop.ts → chatMLFetcher.ts Uses tracer A (Provider A) invoke_agent → chat → execute_tool CLI Background Agent OtelSessionTracker (SDK native) Uses tracer B (Provider B / global) invoke_agent → chat → execute_tool + subagents, permissions, hooks CLI Extension Wrapper copilotcliSession.ts invoke_agent copilotcli (tracer A) Claude Code Agent claudeCodeAgent.ts (synthetic spans) Uses tracer A (Provider A) invoke_agent → execute_tool (flat) Claude Code Subprocess Separate Node process Metrics + events only (no traces) CLI Terminal Process Separate terminal process Full SDK OTel (independent traces) OTelChatDebugLogProvider Listens: onDidCompleteSpan Buckets by copilot_chat.chat_session_id Feeds: Agent Debug Log panel UI ChatDebugFileLoggerService Listens: onDidCompleteSpan Writes: debug-logs//main.jsonl OTLP Exporter (Provider A) BatchSpanProcessor → HTTP/gRPC endpoint Exports foreground + CLI wrapper spans OTLP Exporter (Provider B) BatchSpanProcessor → same HTTP endpoint Exports SDK native spans Grafana / Jaeger / Collector Receives all spans from all exporters Shows full trace hierarchy onDidCompleteSpan BatchSpanProcessor injectCompletedSpan() onEnd() traceparent env vars independent traces metrics+events synthetic spans resolveOTelConfig() Settings: github.copilot.chat.otel.* Env: COPILOT_OTEL_*, OTEL_EXPORTER_* Kill switch: telemetry.telemetryLevel=off OTelConfig
Foreground Agent — Provider A Only All spans created by tracer A (extension's NodeOTelService) Span Hierarchy (tracer A) invoke_agent copilot (INTERNAL) — toolCallingLoop.ts chat gpt-4o (CLIENT) — chatMLFetcher.ts execute_tool readFile (INTERNAL) — toolsService.ts execute_tool runCommand (INTERNAL) — toolsService.ts chat gpt-4o (CLIENT) — chatMLFetcher.ts execute_tool runSubagent (INTERNAL) — toolsService.ts invoke_agent Explore (INTERNAL) — toolCallingLoop.ts All spans have copilot_chat.chat_session_id NodeOTelService Provider A tracer onDidCompleteSpan Debug Panel ✅ OTLP → Grafana ✅ ✅ Fully working Same spans in debug panel and Grafana
CLI Background Agent — Two Providers (Bridge Architecture) Extension root span (tracer A) + SDK native spans (tracer B) + Bridge for debug panel Extension Root (tracer A) invoke_agent copilotcli — copilotcliSession.ts CHAT_SESSION_ID ✅ | CONVERSATION_ID ✅ | SESSION_ID ✅ ↓ traceparent = 00-{traceId}-{spanId}-01 ↓ via otelLifecycle.updateParentTraceContext() SDK spans inherit this traceId → same trace in Grafana ✅ SDK Native Spans (tracer B, same traceId) invoke_agent (CLIENT) — OtelSessionTracker chat claude-opus-4.6-1m (CLIENT) execute_tool report_intent execute_tool task invoke_agent explore (SUBAGENT!) execute_tool grep + permission chat claude-opus-4.6-1m (final response) Bridge injects copilot_chat.chat_session_id on each span ✅ All spans reach debug panel via bridge Provider B (SDK) BatchSpanProcessor → OTLP BridgeSpanProcessor → ext (injected) BridgeSpanProcessor ✅ Injected via internal _spanProcessors ProxyTP._delegate._activeSpanProcessor._spanProcessors.push() NodeOTelService (Provider A) injectCompletedSpan → fire event Debug Panel ✅ (full hierarchy) onDidCompleteSpan OTLP → Grafana ✅ HOW: Bridge injection via OTel v2 internals 1. api.trace.getTracerProvider() → ProxyTracerProvider 2. ._delegate → BasicTracerProvider 3. ._activeSpanProcessor → MultiSpanProcessor 4. ._spanProcessors.push(bridge) — same pattern SDK uses in forceFlush() ✅ Full data flow working • Grafana: full hierarchy via Provider B → BatchSpanProcessor → OTLP • Debug Panel: full hierarchy via Provider B → Bridge → injectCompletedSpan → onDidCompleteSpan • Trace context: SDK spans are children of extension root span (traceparent propagation)
CLI Terminal — Separate Process, Independent Traces Extension only forwards env vars. No bridge. No debug panel visibility. Extension (at terminal creation) deriveCopilotCliOTelEnv(config, {}) → TerminalOptions.env env vars Terminal Process (copilot binary) Own OtelLifecycle + BasicTracerProvider service.name = "github-copilot" invoke_agent → chat → execute_tool Independent root traceId No CHAT_SESSION_ID (not needed) Grafana ✅ Debug Panel ❌ Separate process — spans never reach extension
Claude Code — Synthetic Spans (Only Option) Child process internal spans inaccessible. Extension creates spans from message loop. Extension Synthetic Spans (tracer A) invoke_agent claude — claudeCodeAgent.ts chat claude-sonnet-4 — chatMLFetcher.ts (FREE) execute_tool Read — message loop (PR #4505) chat claude-sonnet-4 — chatMLFetcher.ts execute_tool Edit — message loop (PR #4505) FLAT hierarchy — no subagent nesting, no permissions Claude Code Subprocess (separate process) Own OTel SDK: metrics + events (no traces) claude_code.token.usage, claude_code.cost.usage claude_code.tool_result, claude_code.api_request NodeOTelService Provider A tracer Debug Panel ✅ (flat) OTLP → Grafana ✅ subprocess metrics/events WHY SYNTHETIC (not bridge)? • Claude SDK runs as separate child process • Internal spans are inaccessible from extension • Bridge approach is impossible — synthetic is the only option

Key Design Facts

Two TracerProviders in the Same Process

  • Provider A (NodeOTelService): Created at extension activation. Stores tracer ref. Serves foreground agent, CLI wrapper span, Claude synthetic spans.
  • Provider B (BasicTracerProvider): Created by SDK's OtelLifecycle.initializeOtelSdk(). Replaces A as global. Serves all SDK-native spans.
  • Provider A's stored this._tracer still works after override — OTel tracers are bound to their provider at creation time.

Bridge SpanProcessor

  • Injected into Provider B via internal _spanProcessors.push()
  • Converts ReadableSpan → ICompletedSpanData
  • Injects copilot_chat.chat_session_id from traceId → sessionId map
  • Filters: only forwards spans whose traceId is registered (CLI sessions)
  • Fires IOTelService.injectCompletedSpan() → onDidCompleteSpan

Trace Context Propagation

  • Extension root span → traceparent = 00-{traceId}-{spanId}-01
  • Passed to SDK via otelLifecycle.updateParentTraceContext(sessionId, traceparent)
  • SDK spans inherit the extension's traceId → same trace in Grafana
  • Bridge matches traceId → injects CHAT_SESSION_ID → debug panel buckets correctly

Debug Panel Data Path

  • Foreground: tracer A → TrackedSpanHandle.end() → _onDidCompleteSpan.fire()
  • CLI background: tracer B → Bridge.onEnd() → injectCompletedSpan() → _onDidCompleteSpan.fire()
  • Claude: tracer A → TrackedSpanHandle.end() → _onDidCompleteSpan.fire() (same as foreground)
  • CLI terminal: N/A (separate process, no debug panel visibility)
  • Both consumers: OTelChatDebugLogProvider (trajectory) and ChatDebugFileLoggerService (file)

service.name Values

  • copilot-chat — Extension Provider A spans (foreground, CLI wrapper, Claude synthetic)
  • github-copilot — SDK Provider B spans (CLI native), CLI terminal
  • claude-code — Claude subprocess metrics/events

What Goes Where

  • Grafana: Everything from all OTLP exporters (Provider A + B + terminal + Claude subprocess)
  • Debug Panel: Only spans that fire onDidCompleteSpan with copilot_chat.chat_session_id
  • File Logger: Same as debug panel (listens to same event)