mirror of
https://github.com/microsoft/vscode.git
synced 2026-05-21 15:49:15 +01:00
Sandeep Somavarapu
2e8b995ab1
* agents: add smoke test for Agents Window with mocked LLM server
Adds a smoke test that opens the Agents Window, creates a new session
on a workspace folder, sends a 'hello world' prompt, and verifies the
request reaches a local mock LLM server that returns a canned response.
The test exercises two session types: Copilot CLI and Claude Code. A
third Local-session test is included but marked `it.skip` for now.
Key pieces:
* `test/smoke/src/areas/agents/agents.test.ts` — new smoke suite. Starts
`scripts/chat-simulation/common/mock-llm-server.js` on a random port,
registers a per-test scenario with a distinct reply, and injects
`VSCODE_COPILOT_CHAT_TOKEN` / `GITHUB_PAT` / `IS_SCENARIO_AUTOMATION`
env vars so the Copilot extension's token manager picks up a fake
token whose endpoints.api/proxy point at the mock server.
* `test/automation/src/agents.ts` — new `Agents` workbench helper with
`openCurrentFolderInAgentsWindow`, `switchToAgentsWindow`,
`startNewSession`, `selectSessionType`, `submitNewSessionPrompt`,
`waitForAssistantText`.
* `test/automation/src/code.ts` + `electron.ts` — `LaunchOptions` now
accepts an `extraEnv` map that is merged on top of `process.env`
when spawning the Electron child, so tests can inject env-based mocks
without going through a custom launcher.
* `src/vs/sessions/browser/sessionsSetUpService.ts` —
`shouldSkipSessionsWelcome` now returns `true` whenever
`enableSmokeTestDriver` is set, so the welcome/auth dialog does not
block smoke runs.
* `scripts/chat-simulation/common/mock-llm-server.js` — adds two
models to `EXTRA_MODELS` (`gpt-5.3-codex` for Copilot CLI default,
`claude-sonnet-4.5` for Claude Code), and routes `/v1/messages` to a
new `handleMessagesApi` that streams Anthropic-format SSE
(`message_start` / `content_block_delta` / `message_stop`) which the
Claude Code session type's messages-API parser expects.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix copilot cli test
* skip claude test
* sessions: fix new-session removal regression and rename smoke test
1. Revert the `_refreshSessionCache` filter change from d5747b31c4
back to `adapter instanceof AgentSessionAdapter`. The broadened
`adapter !== this._currentNewSession` check raced with the
unconditional `this._currentNewSession = undefined` in the
`_sendFirstChat*` paths: a late callback from a previous session's
commit would wipe the pointer and the next refresh would evict the
new session's temp adapter, navigating the Agents Window back to the
homepage mid-request.
2. Rename the Agents Window smoke test infrastructure for clarity:
- `test/automation/src/agents.ts` -> `agentsWindow.ts`
(class `Agents` -> `AgentsWindow`,
`workbench.agents` -> `workbench.agentsWindow`)
- `test/smoke/src/areas/agents/agents.test.ts` ->
`areas/agentsWindow/agentsWindow.test.ts`
Verified with 10 consecutive smoke-test loops: 8/10 fully green
(Copilot CLI + Claude + Local), 2 intermittent UI timing flakes
unrelated to the navigation regression.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* agents smoke: address PR #317545 review feedback
- switchToAgentsWindow: replace fixed 2s sleep with waitForElement on
the Agents workbench DOM (`.agent-sessions-workbench`) so the helper
returns as soon as the new window is interactable.
- resolveElectronConfiguration: apply `extraEnv` last, after the
TESTRESOLVER_* assignments in the remote branch, so caller-provided
env vars truly have final precedence.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* agents smoke: skip Agents Window tests for OSS quality
The Copilot extension is not built in OSS CI (it's in excludedExtensions
and its dist/extension.js is only produced by its own esbuild pipeline).
Without it all three session-type providers fail to activate, causing
every Agents Window test to time out.
Skip the suite when quality is OSS, matching the pattern used by
setupExtensionTests and setupLocalizationTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* copilotcli: add proxy endpoint to getAuthInfo for mock server routing
Without `endpoints.proxy`, the SDK's model-fetch calls
(`/models/session`, `/copilot_internal/v2/token`) fall back to
the real GitHub API which rejects the fake HMAC with a 401. This
caused intermittent smoke test failures (1 in 10) because the
Copilot CLI language models never registered, making the chat-setup
readiness gate depend on Claude's model registration timing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* copilotcli: don't cache failed model fetches
When `getAvailableModels` throws (e.g. transient network failure or
HMAC validation error with a proxy), the empty result was permanently
cached in `_availableModels`. Subsequent calls to `getModels()` would
return the cached empty array without retrying, leaving the Copilot CLI
language model provider with zero models for the rest of the session.
Clear `_availableModels` on error so the next call retries the fetch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* agents smoke: reset workspace before opening Agents Window
Earlier smoke test suites (e.g. Tasks) modify .vscode/tasks.json and
leave uncommitted changes. A dirty workspace prevents worktree creation
and triggers the uncommitted-changes confirmation flow which aborts the
Copilot CLI session on builds.
Reset via `git checkout . --quiet` in the before hook, matching the
pattern used by notebook and search smoke tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sandeep Somavarapu <sandy081@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>