vscode/extensions/copilot/docs/monitoring/agent_monitoring.md

Monitoring Agent Usage with OpenTelemetry

Local Copilot Chat can export traces, metrics, and events via OpenTelemetry (OTel) — giving you real-time visibility into extension-host agent interactions, LLM calls, tool executions, and token usage.

This document covers only the local Copilot Chat pipeline configured by github.copilot.chat.otel.*. Agent Host sessions for Copilot, Claude, and Codex run in a separate process, use chat.agentHost.otel.*, and are documented in src/vs/platform/agentHost/OTEL.md.

Standard gen_ai.* signals follow the OTel GenAI Semantic Conventions. Extension-specific signals use the github.copilot.* and legacy copilot_chat.* namespaces. All signals can be exported to OTel-compatible backends such as Jaeger, Grafana, Azure Monitor, Datadog, Honeycomb, and more.

User-perceived first progress

The shared VS Code UI timer also emits the content-free metadata span vscode.chat.user_perceived_time_to_first_progress through this extension's OTel pipeline. Its vscode.chat.user_interaction.timeToFirstProgress attribute is UI submission to meaningful visible text, reasoning, or tool progress plus two animation frames, in milliseconds. This is a rendering approximation, not model TTFT or exact screen paint. Read the attribute, not the export span's duration.

OTel must be enabled, but content capture and VS Code product telemetry need not be enabled. The internal github.copilot.chat.otel.recordUserInteraction command validates and allowlists the shared renderer record. Normal OTel processors export it to the configured destination and SQLite store. The command waits for OTel initialization and exporter flush before acknowledging, so the shared _chat.flushUserInteractionTelemetry command also drains local timing exports. The UI never activates the extension solely to export a timing.

Unsuccessful observations retain their outcome and time to termination, without inventing first-progress latency. Renderer identity and submission ordinal distinguish first submissions from followups without relying on export order. See the shared wire contract for fields, visibility semantics, and export-readiness behavior. Agent Host uses its own exporter rather than this command.

Quick Start

The fastest way to see Copilot Chat traces locally — no cloud account required. This guide uses the Aspire Dashboard, a lightweight container image from Microsoft that provides a trace viewer with a built-in OTLP endpoint. It can be used standalone, without the rest of .NET Aspire.

Prerequisites

• Docker installed
• VS Code with the GitHub Copilot Chat extension

1. Start the Aspire Dashboard

docker run --rm -d \
  -p 18888:18888 \
  -p 4318:18890 \
  --name aspire-dashboard \
  mcr.microsoft.com/dotnet/aspire-dashboard:latest

This exposes the dashboard UI on port 18888 and an OTLP (HTTP) endpoint on port 4318.

2. Configure VS Code

Open Settings (Ctrl+,) and add:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.captureContent": true
}

Note: You can also use environment variables instead of VS Code settings (see Configuration). Environment variables can still override some legacy policy-backed settings; identity capture and managed resource attributes enforce policy precedence. See governed identity capture and the activation limitations.

3. Generate Telemetry

Open Copilot Chat and send any message — for example, ask a question in Agent mode.

4. View Traces

Open http://localhost:18888 → Traces. You'll see invoke_agent spans with nested chat and execute_tool children.

Teardown

docker stop aspire-dashboard

Tip: See the Aspire Dashboard standalone docs for more configuration options.

---

Configuration

VS Code Settings

Open Settings (Ctrl+,) and search for copilot otel:

Setting	Type	Default	Description
github.copilot.chat.otel.enabled	boolean	false	Enable OTel emission
github.copilot.chat.otel.exporterType	string	"otlp-http"	otlp-http, otlp-grpc, console, or file
github.copilot.chat.otel.otlpEndpoint	string	"http://localhost:4318"	OTLP collector endpoint
github.copilot.chat.otel.captureContent	boolean	false	Capture full prompt/response content
github.copilot.chat.otel.captureIdentity	boolean or null	null (off)	Capture authenticated account, OS username, and hostname independently of content. Environment overrides personal preferences; explicit managed policy always wins.
github.copilot.chat.otel.protocol	string	""	OTLP wire protocol: http/json (default), http/protobuf, or grpc
github.copilot.chat.otel.serviceName	string	""	Override the service.name resource attribute
github.copilot.chat.otel.resourceAttributes	object	{}	Extra resource attributes ({ "key": "value" })
github.copilot.chat.otel.headers	object	{}	Extra OTLP exporter headers, applied directly to the exporter ({ "key": "value" })
github.copilot.chat.otel.maxAttributeSizeChars	integer	0	Max characters per OTel content attribute (prompts, tool args/results, hook input/output). 0 (the default) disables truncation so backends with no per-attribute limit get full payloads. Set to a positive value to match your backend's per-attribute size limit — consult your backend's documentation. The value counts JavaScript string characters (UTF-16 code units); for non-ASCII content one character can be multiple UTF-8 bytes on the wire.
github.copilot.chat.otel.outfile	string	""	File path for JSON-lines output
github.copilot.chat.otel.dbSpanExporter.enabled	boolean	false	Persist OTel spans to a local SQLite database for the Chat: Export Agent Traces DB command. Implicitly enables OTel.

Environment Variables

Except for governed identity and managed resource attributes, environment variables retain their existing precedence. When enterprise OTel configuration is recognized through the application-scoped policy defaults, the entire Copilot OTel settings block comes from those policy values and schema defaults. Personal settings.json values are not used to fill omitted fields: headers and resource attributes default to empty maps, not the user's maps. Other VS Code settings are unaffected.

An identity-only managed block (including telemetry.capture.identity: false) is recognizable enterprise OTel configuration. It therefore also replaces personally enabled export, endpoints, headers, and DB export with policy/schema defaults. To keep export enabled under that block, administrators should configure enabled and the intended exporter destination as well. Identity omission remains distinct from denial; it does not by itself prevent personal/environment identity opt-in.

Variable	Default	Description
COPILOT_OTEL_ENABLED	false	Enable OTel. Also enabled when OTEL_EXPORTER_OTLP_ENDPOINT is set.
COPILOT_OTEL_ENDPOINT	—	OTLP endpoint URL (takes precedence over OTEL_EXPORTER_OTLP_ENDPOINT)
OTEL_EXPORTER_OTLP_ENDPOINT	—	Standard OTel OTLP endpoint URL
OTEL_EXPORTER_OTLP_PROTOCOL	http/json	OTLP wire protocol. grpc selects gRPC, http/protobuf selects HTTP protobuf, and other values use HTTP JSON.
COPILOT_OTEL_PROTOCOL	—	Fallback protocol when OTEL_EXPORTER_OTLP_PROTOCOL is unset (grpc, http/protobuf, or HTTP JSON otherwise).
OTEL_SERVICE_NAME	copilot-chat	Service name in resource attributes
OTEL_RESOURCE_ATTRIBUTES	—	Extra resource attributes (key1=val1,key2=val2)
COPILOT_OTEL_CAPTURE_CONTENT	false	Capture full prompt/response content
COPILOT_OTEL_CAPTURE_IDENTITY	false	Opt into identity capture unless explicitly denied by managed policy.
COPILOT_OTEL_MAX_ATTRIBUTE_SIZE_CHARS	0	Override the max character size for OTel content attributes. 0 (default) disables truncation; set to a positive value when your backend has a per-attribute limit. Takes precedence over the maxAttributeSizeChars setting.
COPILOT_OTEL_LOG_LEVEL	info	Min log level: trace, debug, info, warn, error
COPILOT_OTEL_FILE_EXPORTER_PATH	—	Write all signals to this file (JSON-lines)
COPILOT_OTEL_HTTP_INSTRUMENTATION	false	Enable HTTP-level OTel instrumentation
OTEL_EXPORTER_OTLP_HEADERS	—	Auth headers (e.g., Authorization=Bearer token)

Governed identity capture

Managed telemetry.capture.identity is independent of captureContent and lockCaptureContent: neither content control enables or denies identity. An explicit managed false overrides personal settings and environment opt-in. Omission is not a denial and leaves the identity preference available, including when other managed OTel settings replace the personal configuration block.

Across managed sources, the highest-priority telemetry block wins as a whole: native MDM, then server-delivered settings, then the managed-settings file. Omitted identity, resource attributes, and other telemetry leaves are not filled from weaker managed sources. For server and file object sources, an empty block, or one containing only unsupported fields, still replaces the weaker block. Native MDM exposes declared flat keys, not a telemetry object: only an observed declared telemetry key establishes its block. Empty or unknown-only native blocks cannot be observed. Removing all observable telemetry keys (or the server/file block entirely) allows the next managed source to apply. This does not change precedence for unrelated managed settings or the native runtime's own transport. Native delivery also observes the other recognized telemetry.capture.* controls for block selection only; this does not implement those content-capture classes in Local or expose additional Local settings.

When enabled, user.name is the current authenticated provider account name on top-level, subagent, and inline invoke_agent spans. Each invocation reads the authentication service's current session rather than caching an account name in the telemetry service. Account changes and sign-out take effect once the authentication service has refreshed its session. No anonymous identity is invented, and enduser.pseudo.id is unchanged. process.user.name and host.name are resource attributes. Explicit OTEL_RESOURCE_ATTRIBUTES overrides detected OS/host values; managed resourceAttributes overrides environment attributes. Identity attributes are removed when capture is off, even if explicitly supplied as resource attributes. If the OS username cannot be detected (for example, a container user has no passwd entry), a warning is logged and telemetry continues without a detected process.user.name. Hostname capture and explicitly configured resource values remain available.

Compatibility note: Explicit host.name, process.user.name, and user.name resource attributes from environment variables, personal settings, or managed resourceAttributes do not bypass the identity gate. They are removed by default and under an explicit managed denial, not just omitted from automatic detection. Other custom resource attributes are unaffected.

Enabling capture still requires the normal reload/recovery flow. Disabling is checked before subsequent exports and local completion events, including queued spans, logs, metrics, and SQLite export. These checks read cached state refreshed on OTel configuration changes, without resolving settings per span. A denial is latched when the configuration change is observed, even between exports, and stays in effect for that service instance, even if policy later allows capture or is withdrawn; re-enabling capture requires a reload. Already exported or persisted data cannot be recalled, and an export already handed to its transport cannot be cancelled. The nullable default preserves omission versus explicit managed false through the editor's configuration API; it does not enable capture.

This applies to the legacy Local extension-host harness. The native Copilot runtime owns its own managed identity enforcement. The message-content environment bridge does not enable identity or forward account/OS/host values.

Activation

When late enterprise OTel settings turn on external export after Copilot's telemetry service started without it, Copilot can restart the extension hosts for that window to recover. It shows a progress notification before requesting the restart; that notice clears automatically when the host restarts or the attempt ends. Successful recovery is logged without another toast. Restarting also interrupts other extensions in the window. If the restart is unavailable, vetoed, or fails to apply the settings, a warning offers Reload Window instead. User changes and policy withdrawal remain opt-in reloads, except that identity denial is enforced before subsequent exports. Existing exporter selection and legacy environment-variable precedence are unchanged. It uses changes to the application-scoped, policy-backed configuration defaults as a recovery signal, without a new API. Normal personal settings changes do not change those defaults. If a recognizable enterprise OTel block was already present at initialization, later changes only offer a reload, including enabling a previously disabled managed configuration. Automatic recovery additionally requires policy-enabled OTLP export targeting the collector in those defaults. Disabled and DB-only pipelines, unrelated partial policies, and configurations still redirected by environment variables to a different collector or file do not qualify. Policy edits indistinguishable from schema defaults cannot be identified as new policy and retain the opt-in reload behavior. Conflicting environment variables can still prevent recovery, and this does not enforce precedence over environment variables or guarantee telemetry produced before the restart. The default-value signal cannot distinguish a policy consisting entirely of schema-default values from no policy.

There is no periodic polling or restart loop. A startup check and configuration events trigger checks, coalesced by a 500 ms debounce. At most one automatic off-to-on recovery is attempted per workspace and editor session, identified by vscode.env.sessionId. The attempt remains recorded even after success or failure. Later policy updates only offer a reload, deduplicated while stale. If policy first arrives after a later sign-in, that single recovery can happen then rather than immediately at launch. A one-off 15-second grace period allows a requested restart to finish before a still-running host shows the reload fallback.

OTel is off by default with zero overhead. It activates when:

• an applied enterprise policy makes github.copilot.chat.otel.enabled true, or
• COPILOT_OTEL_ENABLED=true, or
• OTEL_EXPORTER_OTLP_ENDPOINT is set, or
• github.copilot.chat.otel.enabled is true, or
• github.copilot.chat.otel.dbSpanExporter.enabled is true (the SDK pipeline must be active to feed the SQLite store).

Commands

Command	Description
Chat: Export Agent Traces DB (github.copilot.chat.otel.exportAgentTracesDB)	Export the local SQLite span database to a .db file. Only available when github.copilot.chat.otel.dbSpanExporter.enabled is true.

---

What Gets Exported

Attribute namespaces & dual-emit policy

Copilot Chat emits OTel attributes under three namespaces:

• gen_ai.* — OTel GenAI Semantic Conventions. Use these whenever a standard key exists.
• github.copilot.* — Canonical Copilot-specific namespace. Prefer this for new dashboards and alerts.
• copilot_chat.* — Original VS Code extension namespace. Several keys (notably copilot_chat.repo.* and gen_ai.usage.reasoning_tokens) are now dual-emitted alongside the github.copilot.* equivalents. Tables below mark these rows as Legacy with a pointer to the preferred key.

Legacy keys continue to emit indefinitely so existing collectors, dashboards, and downstream consumers (Agent Debug Log, Chronicle, SQLite span store) keep working without changes. There is no sunset date.

Traces

Copilot Chat emits a hierarchical span tree for each agent interaction:

invoke_agent copilot                           [~15s]
  ├── chat gpt-4o                              [~3s]  (LLM requests tool calls)
  ├── execute_tool readFile                    [~50ms]
  ├── execute_tool runCommand                  [~2s]
  ├── chat gpt-4o                              [~4s]  (LLM generates final response)
  └── (span ends)

Inline chat uses the same invocation shape, with invoke_agent Inline Chat as the root span and nested chat / execute_tool children.

invoke_agent — wraps the entire agent orchestration (all LLM calls + tool executions).

Attribute	Requirement	Example
gen_ai.operation.name	Required	invoke_agent
gen_ai.provider.name	Required	github
gen_ai.agent.name	Required	copilot
gen_ai.conversation.id	Required	a1b2c3d4-...
gen_ai.request.model	Recommended	gpt-4o
gen_ai.response.model	Recommended	gpt-4o-2024-08-06
gen_ai.usage.input_tokens	Recommended	12500
gen_ai.usage.output_tokens	Recommended	3200
gen_ai.usage.cache_read.input_tokens	When available	8000
gen_ai.usage.cache_creation.input_tokens	When available	4200
github.copilot.agent.type	Always	builtin | custom | plugin
github.copilot.git.repository	When in a repo	https://github.com/microsoft/vscode.git
github.copilot.git.branch	When in a repo	main
github.copilot.git.commit_sha	When in a repo	deadbeef...
github.copilot.github.org	GitHub remotes only	microsoft
copilot_chat.repo.remote_url	Legacy — prefer github.copilot.git.repository	https://github.com/...
copilot_chat.repo.head_branch_name	Legacy — prefer github.copilot.git.branch	main
copilot_chat.repo.head_commit_hash	Legacy — prefer github.copilot.git.commit_sha	deadbeef...
copilot_chat.turn_count	Always	4
error.type	On error	Error
gen_ai.input.messages	Export opt-in (captureContent); retained locally for foreground debugging	[{"role":"user",...}]
gen_ai.output.messages	Export opt-in (captureContent); retained locally for foreground debugging	[{"role":"assistant",...}]
gen_ai.tool.definitions	Export opt-in (captureContent); retained locally for foreground debugging	[{"type":"function",...}]

chat — one span per LLM API call (span kind: CLIENT).

Attribute	Requirement	Example
gen_ai.operation.name	Required	chat
gen_ai.provider.name	Required	github
gen_ai.request.model	Required	gpt-4o
gen_ai.conversation.id	Session correlation (when a session is available)	a1b2c3d4-...
copilot_chat.session_id	Session correlation	a1b2c3d4-...
copilot_chat.chat_session_id	Session correlation	VS Code chat session ID
gen_ai.request.max_tokens	Always	2048
gen_ai.request.temperature	When set	0.1
gen_ai.request.top_p	When set	0.95
copilot_chat.request.max_prompt_tokens	Always	128000
gen_ai.response.id	On response	chatcmpl-abc123
gen_ai.response.model	On response	gpt-4o-2024-08-06
gen_ai.response.finish_reasons	On response	["stop"]
gen_ai.usage.input_tokens	On response	1500
gen_ai.usage.output_tokens	On response	250
gen_ai.usage.cache_read.input_tokens	When available	1200
gen_ai.usage.cache_creation.input_tokens	When available	300
gen_ai.usage.reasoning.output_tokens	When available	512
gen_ai.usage.reasoning_tokens	Legacy — prefer gen_ai.usage.reasoning.output_tokens	512
copilot_chat.time_to_first_token	On response	450
gen_ai.request.stream	Streaming responses	true
gen_ai.response.time_to_first_chunk	Streaming responses (seconds)	0.45
server.address	When available	api.github.com
copilot_chat.debug_name	When available	agentMode
error.type	On error	TimeoutError
gen_ai.input.messages	Opt-in (captureContent)	[{"role":"system",...}]
gen_ai.system_instructions	Opt-in (captureContent)	[{"type":"text",...}]

execute_tool — one span per tool invocation (span kind: INTERNAL).

Attribute	Requirement	Example
gen_ai.operation.name	Required	execute_tool
gen_ai.tool.name	Required	readFile
gen_ai.conversation.id	Session correlation	a1b2c3d4-...
copilot_chat.session_id	Session correlation	a1b2c3d4-...
copilot_chat.chat_session_id	Session correlation	VS Code chat session ID
gen_ai.tool.type	Required	function or extension (MCP tools)
gen_ai.tool.call.id	Recommended	call_abc123
gen_ai.tool.description	When available	Read the contents of a file
github.copilot.tool.parameters.edit_type	Edit tools	create | update | str_replace | insert
github.copilot.tool.parameters.skill_name	When invoking a skill	auto-perf-optimize
github.copilot.tool.parameters.mcp_server_name_hash	MCP tools	SHA-256 hex of server name
github.copilot.tool.parameters.mcp_tool_name	MCP tools	search_issues
github.copilot.tool.parameters.command	Shell tools, opt-in (captureContent)	npm test (truncated to 256 chars)
github.copilot.tool.parameters.file_path	File tools, opt-in (captureContent)	/src/app.ts
github.copilot.tool.parameters.mcp_server_name	MCP tools, opt-in (captureContent)	github
error.type	On error	FileNotFoundError
gen_ai.tool.call.arguments	Export opt-in (captureContent); retained locally (bounded)	{"filePath":"/src/index.ts"}
gen_ai.tool.call.result	Export opt-in (captureContent); retained locally (bounded)	(file contents or summary)

execute_hook — one span per local hook command (span kind: INTERNAL). Hook input and successful output are retained for the Agent Debug Log regardless of captureContent, but require captureContent for primary span export; maxAttributeSizeChars still applies.

Attribute	Requirement	Example
gen_ai.operation.name	Required	execute_hook
gen_ai.conversation.id	Session correlation	a1b2c3d4-...
copilot_chat.hook_type	Required	PreToolUse
copilot_chat.hook_command	Always	./scripts/check.sh
copilot_chat.hook_input	Export opt-in (captureContent); retained locally (bounded)	{"tool_name":"run_in_terminal",...}
copilot_chat.hook_output	Export opt-in (captureContent); retained locally on success (bounded)	ok
copilot_chat.hook_result_kind	On completion	success | error | non_blocking_error
github.copilot.hook.tool_names	When available	["run_in_terminal"]
github.copilot.hook.duration	On completion (seconds)	0.25
github.copilot.hook.decision	On completion	pass | block | non_blocking_error

Metrics

GenAI Convention Metrics

Metric	Type	Unit	Description
gen_ai.client.operation.duration	Histogram	s	LLM API call duration
gen_ai.client.token.usage	Histogram	tokens	Token counts (input/output)
gen_ai.client.operation.time_to_first_chunk	Histogram	s	Time to first streaming chunk
gen_ai.client.operation.time_per_output_chunk	Histogram	s	Inter-chunk latency after the first chunk

gen_ai.client.operation.duration attributes:

Attribute	Description
gen_ai.operation.name	Operation type (e.g., chat)
gen_ai.provider.name	Provider (e.g., github, anthropic)
gen_ai.request.model	Requested model
gen_ai.response.model	Resolved model (if different)
server.address	Server hostname
server.port	Server port
error.type	Error class (if failed)

gen_ai.client.token.usage attributes:

Attribute	Description
gen_ai.operation.name	Operation type
gen_ai.provider.name	Provider name
gen_ai.token.type	input or output
gen_ai.request.model	Requested model
gen_ai.response.model	Resolved model
server.address	Server hostname

gen_ai.client.operation.time_to_first_chunk / gen_ai.client.operation.time_per_output_chunk attributes:

Attribute	Description
gen_ai.operation.name	Operation type (e.g., chat)
gen_ai.provider.name	Provider (e.g., github, anthropic, gemini)
gen_ai.request.model	Requested model
gen_ai.response.model	Resolved model (when known)

Both metrics are tagged with gen_ai.response.model for per-model slicing. time_per_output_chunk is emitted only on the primary GitHub streaming path; BYOK providers (Anthropic, Gemini) emit time_to_first_chunk only, as they do not expose per-chunk arrival timing.

Extension-Specific Metrics

Metric	Type	Unit	Description
copilot_chat.tool.call.count	Counter	calls	Tool invocations by name and success
copilot_chat.tool.call.duration	Histogram	ms	Tool execution latency
copilot_chat.agent.invocation.duration	Histogram	s	Agent mode end-to-end duration
copilot_chat.agent.turn.count	Histogram	turns	LLM round-trips per agent invocation
copilot_chat.session.count	Counter	sessions	Chat sessions started
copilot_chat.time_to_first_token	Histogram	s	Time to first SSE token

copilot_chat.tool.call.count attributes: gen_ai.tool.name, success (boolean)

copilot_chat.tool.call.duration attributes: gen_ai.tool.name

copilot_chat.agent.invocation.duration attributes: gen_ai.agent.name

copilot_chat.agent.turn.count attributes: gen_ai.agent.name

copilot_chat.time_to_first_token attributes: gen_ai.request.model

Agent Activity & Outcome Metrics

These metrics track activity and outcomes reported through the Copilot Chat extension's OTel service.

Metric	Type	Unit	Description
copilot_chat.edit.acceptance.count	Counter	edits	Edit accept/reject decisions (inline chat, chat editing, hunk-level)
copilot_chat.chat_edit.outcome.count	Counter	edits	File-level chat editing session outcomes (accepted/rejected/saved)
copilot_chat.lines_of_code.count	Counter	lines	Lines of code added/removed by accepted agent edits
copilot_chat.edit.survival.four_gram	Histogram	ratio (0-1)	4-gram text similarity survival score
copilot_chat.edit.survival.no_revert	Histogram	ratio (0-1)	No-revert survival score
copilot_chat.user.action.count	Counter	actions	User engagement: copy, insert, apply, followup
copilot_chat.user.feedback.count	Counter	votes	Thumbs up/down on chat responses
copilot_chat.agent.edit_response.count	Counter	responses	Agent edit responses by success/error
copilot_chat.agent.summarization.count	Counter	events	Context summarization outcomes (applied/failed)
copilot_chat.pull_request.count	Counter	PRs	Pull requests created via CLI agent
copilot_chat.cloud.session.count	Counter	sessions	Cloud/remote agent sessions by partner
copilot_chat.cloud.operation.count	Counter	operations	Cloud task operation outcomes (create, fetch, follow-up, PR)
copilot_chat.cloud.operation.duration	Histogram	ms	Cloud task operation latency
copilot_chat.cloud.error.count	Counter	errors	Cloud task operation failures by operation and error type

copilot_chat.edit.acceptance.count attributes: copilot_chat.edit.source (inline_chat/chat_editing/chat_editing_hunk/apply_patch/replace_string/code_mapper), copilot_chat.edit.outcome (accepted/rejected), copilot_chat.language_id (optional)

copilot_chat.chat_edit.outcome.count attributes: copilot_chat.edit.source, copilot_chat.edit.outcome (accepted/rejected/saved), copilot_chat.language_id (optional), copilot_chat.has_remaining_edits (optional)

copilot_chat.lines_of_code.count attributes: type (added/removed), copilot_chat.language_id (optional)

copilot_chat.edit.survival.four_gram attributes: copilot_chat.edit.source, copilot_chat.time_delay_ms

copilot_chat.edit.survival.no_revert attributes: copilot_chat.edit.source, copilot_chat.time_delay_ms

copilot_chat.user.action.count attributes: action (copy/insert/apply/followup)

copilot_chat.user.feedback.count attributes: rating (positive/negative)

copilot_chat.agent.edit_response.count attributes: outcome (success/error)

copilot_chat.agent.summarization.count attributes: outcome (applied/failed)

copilot_chat.cloud.session.count attributes: partner_agent (copilot/claude/codex)

copilot_chat.cloud.operation.count attributes: operation (createSession/fetchSessionList/fetchContent/fetchEvents/pollUpdate/followUp/createPullRequest/sessionActivated), success

copilot_chat.cloud.operation.duration attributes: operation

copilot_chat.cloud.error.count attributes: operation, error.type (low-cardinality classifier, e.g. http_500)

Events

gen_ai.client.inference.operation.details

Emitted after each LLM API call with full inference metadata.

Attribute	Description
gen_ai.operation.name	Always chat
gen_ai.request.model	Requested model
gen_ai.response.model	Resolved model
gen_ai.response.id	Response ID
gen_ai.response.finish_reasons	Stop reasons (e.g., ["stop"])
gen_ai.usage.input_tokens	Input token count
gen_ai.usage.output_tokens	Output token count
gen_ai.request.temperature	Temperature (if set)
gen_ai.request.max_tokens	Max tokens (if set)
error.type	Error class (if failed)
gen_ai.input.messages	Full prompt messages (captureContent only)
gen_ai.system_instructions	System prompt (captureContent only)
gen_ai.tool.definitions	Tool schemas (captureContent only)

copilot_chat.session.start

Emitted when a new chat session begins (top-level agent invocations only, not subagents).

Attribute	Description
session.id	Session identifier
gen_ai.request.model	Initial model
gen_ai.agent.name	Chat participant name

copilot_chat.tool.call

Emitted when a tool invocation completes.

Attribute	Description
gen_ai.tool.name	Tool name
duration_ms	Execution time in milliseconds
success	true or false
error.type	Error class (if failed)

copilot_chat.agent.turn

Emitted for each LLM round-trip within an agent invocation.

Attribute	Description
turn.index	Turn number (0-indexed)
gen_ai.usage.input_tokens	Input tokens this turn
gen_ai.usage.output_tokens	Output tokens this turn
tool_call_count	Number of tool calls this turn

Agent Activity & Outcome Events

These events provide drill-down detail for the agent activity metrics above. They are emitted as OTel log records.

copilot_chat.edit.feedback

Emitted when a user accepts or rejects a file-level edit from the agent.

Attribute	Description
outcome	accepted or rejected
language_id	Language of the edited file
participant	Chat participant that proposed the edit
request_id	Chat request identifier
edit_surface	agent or inline_chat
has_remaining_edits	Whether unreviewed edits remain
is_notebook	Whether the file is a notebook

copilot_chat.edit.hunk.action

Emitted when a user accepts or rejects an individual hunk.

Attribute	Description
outcome	accepted or rejected
language_id	Language of the edited file
request_id	Chat request identifier
line_count	Total lines in the hunk
lines_added	Lines added
lines_removed	Lines removed

copilot_chat.inline.done

Emitted when an inline chat edit is accepted or rejected.

Attribute	Description
accepted	true or false
language_id	Language of the edited file
edit_count	Number of edits suggested
edit_line_count	Total lines across all edits
reply_type	How the response was shown
is_notebook	Whether the document is a notebook

copilot_chat.edit.survival

Emitted at intervals (5s, 30s, 2min, 5min, 10min, 15min) after an edit is accepted, measuring how much of the AI-generated code survives.

Attribute	Description
edit_source	apply_patch, replace_string, code_mapper, or inline_chat
survival_rate_four_gram	0-1 ratio of AI edit still present (4-gram similarity)
survival_rate_no_revert	0-1 ratio of edit ranges not reverted
time_delay_ms	Milliseconds since edit acceptance
did_branch_change	Whether git branch changed (ignore if true)
request_id	Chat request identifier

copilot_chat.user.feedback

Emitted when a user votes on a chat response (thumbs up/down).

Attribute	Description
rating	positive or negative
participant	Chat participant name
conversation_id	Conversation session ID
request_id	Chat request identifier

copilot_chat.cloud.session.invoke

Emitted when a cloud/remote agent session is started.

Attribute	Description
partner_agent	copilot, claude, or codex
model	Model identifier
request_id	Chat request identifier

Resource Attributes

All signals carry:

Attribute	Value
service.name	copilot-chat (override via the github.copilot.chat.otel.serviceName setting, OTEL_SERVICE_NAME, or enterprise policy)
service.version	Extension version
session.id	Unique per VS Code window

Add custom resource attributes with OTEL_RESOURCE_ATTRIBUTES:

export OTEL_RESOURCE_ATTRIBUTES="team.id=platform,department=engineering"

These custom attributes are included in all traces, metrics, and events, allowing you to:

• Filter metrics by team or department
• Create team-specific dashboards and alerts
• Track usage across organizational boundaries

Note: OTEL_RESOURCE_ATTRIBUTES uses comma-separated key=value pairs. The extension splits on commas, trims surrounding whitespace, and does not percent-decode values. Use the github.copilot.chat.otel.resourceAttributes object setting when a value needs to contain a comma.

The governed identity keys (user.name, process.user.name, host.name) are excluded even when explicitly configured unless identity capture is allowed. See Governed identity capture.

---

Content Capture

The captureContent setting controls content on individual LLM chat spans and inference events. It also gates known content attributes on spans and span events sent through the Local harness's primary exporters: OTLP, file, and console. When off, the export boundary removes input/output messages, system instructions, tool definitions, tool arguments/results, user requests, reasoning, prompt context/instructions, markdown content, hook input/output, and generic content/toolDefinitions attributes. Span events can remain without their content attributes.

Foreground invoke_agent, execute_tool, and execute_hook spans still retain content needed by the Agent Debug Log and the separate local SQLite exporter, regardless of this setting. Primary file export is not the SQLite debug exporter. Identity filtering applies to both paths independently.

To include these content attributes in primary exports and capture full LLM request and response content, add to your VS Code settings:

{
  "github.copilot.chat.otel.captureContent": true
}

This additionally populates content attributes on chat spans and inference events, including:

Attribute	Content
gen_ai.input.messages	Full LLM input messages (JSON)
gen_ai.output.messages	Full LLM response messages (JSON)
gen_ai.system_instructions	System prompt
gen_ai.tool.definitions	Tool schemas

Content attributes are not truncated by default. Set github.copilot.chat.otel.maxAttributeSizeChars to a positive value when the backend imposes a per-attribute limit.

Warning: This is filtering of known content attributes, not universal payload sanitization. Other metadata, span names, status messages, or unrecognized attributes may still contain sensitive information. Local debug content retention is separate. Enabling captureContent includes prompt, response, and tool content in primary exports. Configure export only for trusted environments.

---

Example Configurations

OTLP/gRPC:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-grpc",
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4317"
}

Remote collector with authentication:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.otlpEndpoint": "https://collector.example.com:4318"
}

Note: Authentication headers can be set via the github.copilot.chat.otel.headers setting (a { "key": "value" } map applied directly to the exporter) or the OTEL_EXPORTER_OTLP_HEADERS environment variable, and can be mandated by enterprise policy. See Environment Variables.

File-based output (offline / CI):

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "file",
  "github.copilot.chat.otel.outfile": "/tmp/copilot-otel.jsonl"
}

The file contains newline-delimited JSON records for spans, logs, and metrics; it is not an OTLP JSON payload. Span records contain public span data, including trace and span IDs, parent context, attributes, events, links, resource attributes, instrumentation scope, status, and dropped-data counts. Trace state is serialized as a string. Span startTime, endTime, and duration use [seconds, nanoseconds] pairs, preserving nanosecond precision.

The exporter preserves attributes already captured by instrumentation; see Content Capture for what those attributes can contain. Logs and metrics retain their existing SDK JSON representation. Serialization failures are reported as failed exports rather than written as {} placeholders, and a batch that cannot be serialized is not partially written.

Console output (quick debugging):

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "console"
}

---

Subagent Trace Propagation

When an agent invokes a subagent (e.g., via the runSubagent tool), Copilot Chat automatically propagates the trace context so the subagent's invoke_agent span is parented to the calling agent's execute_tool span. This produces a connected trace tree:

invoke_agent copilot                           [~30s]
  ├── chat gpt-4o                              [~3s]
  ├── execute_tool runSubagent                 [~20s]
  │   └── invoke_agent Explore                 [~18s]   ← child via trace context
  │       ├── chat gpt-4o                      [~2s]
  │       ├── execute_tool searchFiles         [~200ms]
  │       ├── execute_tool readFile            [~50ms]
  │       └── chat gpt-4o                      [~3s]
  ├── chat gpt-4o                              [~4s]
  └── (span ends)

This propagation works across async boundaries — the parent's trace context is stored when runSubagent starts and retrieved when the subagent begins its invoke_agent span.

---

Interpreting the Data

Traces — Visualize the full agent execution in Jaeger or Grafana Tempo. Each invoke_agent span contains child chat and execute_tool spans, making it easy to identify bottlenecks and debug failures. Foreground subagent invocations appear as nested invoke_agent spans under execute_tool runSubagent.

Metrics — Track token usage trends by model and provider, monitor tool success rates via copilot_chat.tool.call.count, and watch perceived latency with copilot_chat.time_to_first_token. Agent activity metrics (copilot_chat.edit.acceptance.count, copilot_chat.edit.survival.four_gram, copilot_chat.lines_of_code.count) power accept rate and edit survival dashboards. All metrics carry the same resource attributes (service.name, service.version, session.id) for consistent filtering.

Events — copilot_chat.session.start tracks session creation. copilot_chat.tool.call events provide per-invocation timing and error details. copilot_chat.edit.feedback and copilot_chat.edit.survival events enable drill-down into which edits were accepted/rejected and how code survival varies by edit source. copilot_chat.user.feedback links thumbs-up/down votes to specific conversations for quality investigation. gen_ai.client.inference.operation.details gives the full LLM call record including token usage and, when content capture is enabled, the complete prompt/response messages. Use gen_ai.conversation.id to correlate all signals belonging to the same session.

---

Initialization & Buffering

The OTel SDK is loaded asynchronously via dynamic imports to avoid blocking extension startup. Events emitted before initialization completes are buffered (up to 1,000 items) and replayed once the SDK is ready. If initialization fails, buffered events are discarded and all subsequent calls become no-ops — the extension continues to function normally.

First successful span export is logged to the console ([OTel] First span batch exported successfully via ...) to confirm end-to-end connectivity.

---

Backend Setup Guides

Copilot Chat's OTel data works with any OTLP-compatible backend. This section provides step-by-step setup guides for recommended backends.

Aspire Dashboard

See Quick Start above for setup. The Aspire Dashboard is the simplest option — a single Docker container with a built-in OTLP endpoint and trace viewer. No cloud account or collector needed.

OTel Collector + Azure Application Insights

Azure Application Insights ingests OTel traces, metrics, and logs through an OTel Collector with the azuremonitor exporter. This repo includes a ready-to-use collector setup in docs/monitoring/.

1. Create an Application Insights resource:

1. Go to the Azure Portal.
2. Click Create a resource → search Application Insights → Create.
3. Choose your subscription, resource group, name, and region → Review + Create → Create.
4. Once deployed, go to the resource → Overview → copy the Connection String.

2. Start the OTel Collector:

export APPLICATIONINSIGHTS_CONNECTION_STRING="InstrumentationKey=...;IngestionEndpoint=..."

cd docs/monitoring
docker compose up -d

Verify the collector is healthy:

# Should return 200
curl -s -o /dev/null -w "%{http_code}" http://localhost:4328/v1/traces \
  -X POST -H "Content-Type: application/json" -d '{"resourceSpans":[]}'

3. Configure VS Code:

Open Settings (Ctrl+,) and add:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-http",
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4328"
}

Optionally, to capture full prompt/response content:

{
  "github.copilot.chat.otel.captureContent": true
}

Warning: Content capture includes prompts, code, and file contents. Only enable in trusted environments.

4. Generate telemetry — Open Copilot Chat and send any message (e.g., use Agent mode).

5. Verify data:

• Jaeger (local): Open http://localhost:16687, select service copilot-chat, click Find Traces.
• App Insights (Azure): Go to your Application Insights resource → Transaction search → filter by "Trace" or "Request".

Run this query in Application Insights → Logs to confirm:

traces
| where timestamp > ago(1h)
| where message contains "GenAI" or message contains "copilot_chat"
| project timestamp, message, customDimensions
| order by timestamp desc

For metrics (may take 5–10 minutes to appear):

customMetrics
| where timestamp > ago(1h)
| where name startswith "gen_ai" or name startswith "copilot_chat"
| summarize avg(value), count() by name

Collector config (docs/monitoring/otel-collector-config.yaml):

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  azuremonitor:
    connection_string: "${APPLICATIONINSIGHTS_CONNECTION_STRING}"
  debug:
    verbosity: basic

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [azuremonitor, debug]
    metrics:
      receivers: [otlp]
      exporters: [azuremonitor, debug]

Note: The docker-compose maps ports to 4328/4327 on the host to avoid conflicts. Adjust in docker-compose.yaml if needed. Add additional exporters (e.g., otlphttp/jaeger) to fan out to multiple backends. See docs/monitoring/otel-collector-config.yaml for the full config including batch processor and logs pipeline.

Jaeger

Jaeger is an open-source distributed tracing platform. It accepts OTLP directly — no collector needed.

1. Start Jaeger:

docker run -d --name jaeger -p 16686:16686 -p 4318:4318 jaegertracing/jaeger:latest

2. Configure VS Code:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4318"
}

3. Verify: Open http://localhost:16686, select service copilot-chat, and click Find Traces.

Langfuse

Langfuse is an open-source LLM observability platform with native OTLP ingestion and support for OTel GenAI Semantic Conventions. See the Langfuse docs for full details on capabilities and limitations.

Setup:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:3000/api/public/otel",
  "github.copilot.chat.otel.captureContent": true
}

Set the auth header with the github.copilot.chat.otel.headers setting or the OTEL_EXPORTER_OTLP_HEADERS environment variable. For example:

{
  "github.copilot.chat.otel.headers": {
    "Authorization": "Basic <base64-public-key-and-secret-key>"
  }
}

Replace <public-key> and <secret-key> with your Langfuse API keys from Settings → API Keys.

Verify: Open Langfuse → Traces. You should see invoke_agent traces with nested chat and execute_tool spans.

Other Backends

Any OTLP-compatible backend works with Copilot Chat's OTel output. Some options:

Backend	Description
Jaeger	Open-source distributed tracing platform
Grafana Tempo + Prometheus	Open-source traces + metrics stack

Refer to each backend's documentation for OTLP ingestion setup.

---

Security & Privacy

• Export is off by default. No OTel data is sent to an external exporter unless explicitly enabled. When disabled, the OTel SDK is not loaded; a lightweight in-memory service still records data needed by the Agent Debug Log.
• Content export and local debug retention are separate. Known span and span-event content attributes require captureContent in primary OTLP, file, and console exports. Foreground agent, tool, and hook spans retain content for local debugging even when that setting is off. This is not universal sanitization; review Content Capture before enabling export.
• Treat exported content as sensitive. User messages, file contents, commands, tool payloads, and hook data can contain personal or confidential information.
• User-configured endpoints. Data goes only where you point it — no phone-home behavior.
• Dynamic imports only. OTel SDK packages are loaded on-demand, ensuring zero bundle impact when disabled.