* agents: add smoke test for Agents Window with mocked LLM server
Adds a smoke test that opens the Agents Window, creates a new session
on a workspace folder, sends a 'hello world' prompt, and verifies the
request reaches a local mock LLM server that returns a canned response.
The test exercises two session types: Copilot CLI and Claude Code. A
third Local-session test is included but marked `it.skip` for now.
Key pieces:
* `test/smoke/src/areas/agents/agents.test.ts` — new smoke suite. Starts
`scripts/chat-simulation/common/mock-llm-server.js` on a random port,
registers a per-test scenario with a distinct reply, and injects
`VSCODE_COPILOT_CHAT_TOKEN` / `GITHUB_PAT` / `IS_SCENARIO_AUTOMATION`
env vars so the Copilot extension's token manager picks up a fake
token whose endpoints.api/proxy point at the mock server.
* `test/automation/src/agents.ts` — new `Agents` workbench helper with
`openCurrentFolderInAgentsWindow`, `switchToAgentsWindow`,
`startNewSession`, `selectSessionType`, `submitNewSessionPrompt`,
`waitForAssistantText`.
* `test/automation/src/code.ts` + `electron.ts` — `LaunchOptions` now
accepts an `extraEnv` map that is merged on top of `process.env`
when spawning the Electron child, so tests can inject env-based mocks
without going through a custom launcher.
* `src/vs/sessions/browser/sessionsSetUpService.ts` —
`shouldSkipSessionsWelcome` now returns `true` whenever
`enableSmokeTestDriver` is set, so the welcome/auth dialog does not
block smoke runs.
* `scripts/chat-simulation/common/mock-llm-server.js` — adds two
models to `EXTRA_MODELS` (`gpt-5.3-codex` for Copilot CLI default,
`claude-sonnet-4.5` for Claude Code), and routes `/v1/messages` to a
new `handleMessagesApi` that streams Anthropic-format SSE
(`message_start` / `content_block_delta` / `message_stop`) which the
Claude Code session type's messages-API parser expects.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix copilot cli test
* skip claude test
* sessions: fix new-session removal regression and rename smoke test
1. Revert the `_refreshSessionCache` filter change from d5747b31c4
back to `adapter instanceof AgentSessionAdapter`. The broadened
`adapter !== this._currentNewSession` check raced with the
unconditional `this._currentNewSession = undefined` in the
`_sendFirstChat*` paths: a late callback from a previous session's
commit would wipe the pointer and the next refresh would evict the
new session's temp adapter, navigating the Agents Window back to the
homepage mid-request.
2. Rename the Agents Window smoke test infrastructure for clarity:
- `test/automation/src/agents.ts` -> `agentsWindow.ts`
(class `Agents` -> `AgentsWindow`,
`workbench.agents` -> `workbench.agentsWindow`)
- `test/smoke/src/areas/agents/agents.test.ts` ->
`areas/agentsWindow/agentsWindow.test.ts`
Verified with 10 consecutive smoke-test loops: 8/10 fully green
(Copilot CLI + Claude + Local), 2 intermittent UI timing flakes
unrelated to the navigation regression.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* agents smoke: address PR #317545 review feedback
- switchToAgentsWindow: replace fixed 2s sleep with waitForElement on
the Agents workbench DOM (`.agent-sessions-workbench`) so the helper
returns as soon as the new window is interactable.
- resolveElectronConfiguration: apply `extraEnv` last, after the
TESTRESOLVER_* assignments in the remote branch, so caller-provided
env vars truly have final precedence.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* agents smoke: skip Agents Window tests for OSS quality
The Copilot extension is not built in OSS CI (it's in excludedExtensions
and its dist/extension.js is only produced by its own esbuild pipeline).
Without it all three session-type providers fail to activate, causing
every Agents Window test to time out.
Skip the suite when quality is OSS, matching the pattern used by
setupExtensionTests and setupLocalizationTests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* copilotcli: add proxy endpoint to getAuthInfo for mock server routing
Without `endpoints.proxy`, the SDK's model-fetch calls
(`/models/session`, `/copilot_internal/v2/token`) fall back to
the real GitHub API which rejects the fake HMAC with a 401. This
caused intermittent smoke test failures (1 in 10) because the
Copilot CLI language models never registered, making the chat-setup
readiness gate depend on Claude's model registration timing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* copilotcli: don't cache failed model fetches
When `getAvailableModels` throws (e.g. transient network failure or
HMAC validation error with a proxy), the empty result was permanently
cached in `_availableModels`. Subsequent calls to `getModels()` would
return the cached empty array without retrying, leaving the Copilot CLI
language model provider with zero models for the rest of the session.
Clear `_availableModels` on error so the next call retries the fetch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* agents smoke: reset workspace before opening Agents Window
Earlier smoke test suites (e.g. Tasks) modify .vscode/tasks.json and
leave uncommitted changes. A dirty workspace prevents worktree creation
and triggers the uncommitted-changes confirmation flow which aborts the
Copilot CLI session on builds.
Reset via `git checkout . --quiet` in the before hook, matching the
pattern used by notebook and search smoke tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Sandeep Somavarapu <sandy081@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Bump version to 1.117.0
* npm i
* wait to do engine version bump
* Revert "wait to do engine version bump"
This reverts commit 9db1c0feb6.
* Add Copilot extension tests to Linux/Windows Electron integration test runs
* Remove failing step that we moved to the main build
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Add argument parsing, suite filtering, and grep support to integration test scripts
- Add --run, --runGlob, --grep, --suite, and --help argument parsing
- --suite selects extension host test suites (comma-separated, glob patterns)
- --grep forwards test name filter to all runners via MOCHA_GREP env var
- Validate --suite filter matches at least one known suite
- Add MOCHA_GREP support to testrunner.js, CSS and HTML test runners
- Seed user settings to suppress dock bounce notifications
- Always apply *.integrationTest.js glob for node.js tests
- Add integration-tests skill documentation
* Address Copilot review feedback
- Quote cd $ROOT, rm -rf $VSCODEUSERDATADIR, rmdir %VSCODEUSERDATADIR%
- Quote --runGlob pattern to prevent premature glob expansion
- Use GREP_ARGS array for safe grep forwarding in .sh
- Use conditional call with proper quoting for grep in .bat
- Deduplicate suite list into KNOWN_SUITES variable
- Remove unused EXTRA_ARGS and ARGS variables from .bat
* Fix Windows CI: remove unnecessary enabledelayedexpansion
The original script used plain 'setlocal'. Adding 'enabledelayedexpansion'
may affect path resolution behavior on Windows CI. Since no delayed
expansion (\!var\!) syntax is used, revert to the original 'setlocal'.
* Fix Windows CI: capture %~dp0 before call :label corrupts it
In Windows batch, 'call :label' can change what %~dp0 resolves to.
Our should_run_suite subroutine uses 'call :should_run_suite', which
caused %~dp0 to resolve to the wrong directory for extension paths
that appear after the subroutine call. Capture the script directory
once at startup into %SCRIPT_DIR% and use it everywhere.
- Migrates to use AHP types that are synced via `npx tsx scripts/sync-agent-host-protocol.ts`
- One big churn was migrating out of URIs as rich objects in the protocol.
We can't really shove our own `$mid`-type objects in there. I also explored doing a
generated translation layer but I had trouble getting one I was happy with.
- This tightens up some type safety in general and fixes some areas where vscode had
silently/sloppily diverged from the protocol types.
The test/e2e/ directory was too deep for TypeScript to resolve imports
back to src/vs/. Move to test/ (one level up) where imports work.
E2E runner scripts (.cjs) and scenarios stay in test/e2e/.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keeps all E2E test code (mocks, runners, scenarios, extensions) in one
place, separate from production Sessions code. Updates import paths,
server path, and ESLint config.
Note: ESLint import-patterns warnings remain because the relative
imports from test/e2e/ depth don't match the restriction glob syntax.
This is cosmetic — the imports are valid TypeScript.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New files:
- web.test.ts: TestSessionsBrowserMain with MockChatEntitlementService
- web.test.factory.ts: Factory using TestSessionsBrowserMain
- sessions.web.test.internal.ts: Test entry point
Server uses test entry point when --mock, which injects a mock
IChatEntitlementService returning ChatEntitlement.Free with
installed=true, so Sessions thinks the user is signed in.
Also adds ESLint import patterns for sessions web entry points.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pre-sets the workspace to github-remote-file://mock-repo/ so the
Sessions UI starts with a folder already open, backed by the mock
file system provider from the E2E extension.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New extension at extensions/sessions-e2e-mock/ provides:
- Mock GitHub auth provider (fake token, skips sign-in)
- Mock chat participant (canned responses based on input keywords)
- Mock file system for github-remote-file:// (in-memory files)
Server loads the extension when --mock flag is passed. The generate
and test runners both use --mock automatically.
New npm scripts:
- serve:mock — opens Sessions in browser with mocks loaded
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The serve script now opens with ?skip-sessions-welcome to bypass
the sign-in overlay during manual testing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Created new YAML files to define the structure and components of the session management interface.
- Included various toolbars, buttons, and alerts to enhance user interaction.
- Implemented features such as session actions, agent sessions, and folder picker options.
- Ensured accessibility with appropriate cursor pointers and disabled states for buttons.
* Run our build scripts directly as typescript #277567
Follow up on #276864
For #277526
* Remove a few more ts-node references
* Fix linux and script reference
* Remove `_build-script` ref
* Fix script missing closing quote
* use type only import
* Fix export
* Make sure to run copy-policy-dto
* Make sure we run the copy-policy-dto script
* Enable `verbatimModuleSyntax`
* Pipelines fixes
* Try adding explicit ext to path
* Fix bad edit
* Revert extra `--`
---------
Co-authored-by: João Moreno <joaomoreno@users.noreply.github.com>
For #269213
This adds a new eslint rule for `as any` and `<any>({... })`. We'd like to remove almost all of these, however right now the first goal is to prevent them in new code. That's why with this first PR I simply add `eslint-disable` comments for all breaks
Trying to get this change in soon after branching off for release to hopefully minimize disruption during debt week work
* Revert "Revert "tests - enable API tests with test runner" (#241146)"
This reverts commit 105d4af21c.
* update
* try to fix
* fix: set extension development path for API tests
---------
Co-authored-by: Connor Peet <connor@peet.io>
Sandeep Somavarapu
Connor Peet
Paul
Alex Ross
Peng Lyu
Rob Lourens
Osvaldo Ortega
Robo
Henning Dieterichs
Benjamin Pasero
Dmitriy Vasyura
Daniel Imms
Matt Bierner
Ben Villalobos
Benjamin Pasero