mirror of
https://github.com/microsoft/vscode.git
synced 2026-08-18 13:17:05 +01:00
5ce4a53da0
* agent host e2e: reuse one replay server per suite for Copilot The agent host e2e replay suite restarted the whole server (and re-spawned the provider SDK/CLI subprocess) for every test. Profiling showed the dominant per-test cost is server fork (~0.5s) plus the provider SDK client cold start (~0.5s, an ~8MB CLI bundle loaded on the Electron-as-node binary), with session teardown effectively free. Reuse one server across the suite for providers that can be returned to a clean state via disposeSession, swapping the per-test replay fixture on the in-process CapiReplayProxy without restarting (the proxy URL the agent host was forked against stays fixed): - CapiReplayProxy: add resetForReplay (re-point at a new fixture, clear replay buckets + cache-miss log), assertNoCacheMisses (verify strict replay without stopping), and close (drop the socket without the strict check). stop() now composes these. - Add AgentHostE2EServerLease to encapsulate per-test vs shared strategies, gated by a new supportsSharedReplayServer provider flag (replay only; recording always uses a fresh proxy + fixture per test). - Enable it for Copilot. Claude stays per-test: its mid-turn dispose can wedge the shared agent host process, which per-test kill-and-refork isolation guards against. - Drain the permission test's post-tool continuation to turnComplete in all modes so the session ends cleanly before dispose (returning mid-turn would wedge the reused client). Copilot suite drops from ~29s to ~14s mocha time. Claude and Codex are unchanged (per-test path). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agent host e2e: address Copilot review feedback - CapiReplayProxy: route the constructor's fixture-existence check and error message through this._fixturePath so all fixture-path access has a single source of truth. - AgentHostE2EServerLease.acquire: fail fast with a clear error instead of a non-null assertion when a shared server unexpectedly has no capiReplay proxy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agent host e2e: document the shared replay server in the README Add a "Server lifecycle" section covering the per-test vs shared strategies, how AgentHostE2EServerLease and CapiReplayProxy.resetForReplay swap fixtures without restarting (in-process, no IPC), why sharing is opt-in per provider (Copilot yes, Claude no), and the "don't return mid-turn" rule for authors. Also add the supportsSharedReplayServer gate row, list the lease in the components table, and add a troubleshooting entry for cross-test state leaks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agent host e2e: enable the shared replay server for Claude and Codex Investigated the long-standing "Claude mid-turn dispose wedges the agent host" caveat that kept server reuse Copilot-only. It dates from the live real-SDK era (real streaming turns actually in flight). In today's deterministic replay suite there is no mid-turn dispose: the abort test is record-only and every turn drains to `turnComplete` before teardown. Deliberately reproducing the old failure (reverting the permission test's drain) showed it is not a process wedge at all but a strict cache miss: an undrained turn's continuation HTTP call fires after the fixture is swapped and lands in the next test's window as an unrecorded call. The proxy detects it loudly. The cure is exactly the drain already in place, so with every turn drained there is nothing to leak. Enable `supportsSharedReplayServer` for Claude and Codex (verified green and stable across repeated runs; Claude ~16s -> ~9s, Codex likewise). Correct the drain rationale and the README/lease/flag comments to describe the real mechanism (leaked async traffic) instead of the "wedge" folklore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agent host e2e: drop supportsSharedReplayServer, share by default in replay The flag was true for all three providers, so it modeled the wrong thing: a per-provider capability, when the real requirement is a per-test invariant (drain your turns) that every provider already meets. Server reuse is really a property of replay mode, gated only by recording (which needs a fresh proxy per fixture). Remove the flag and make the lease share by default in replay (`_shared = !RECORD`). No behavior change (all three suites already reused the server); this just deletes dead config and the misleading "some providers can't share" framing. Single-test isolation for diagnosing a cross-test leak is still available via `--grep`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>