Robo
8748be1f1a
* feat: update to Electron v42
* chore: drop support for arm 32-bit server
* chore: update types/node to v24.x
* chore: temporarily lock the target version for build/
* chore: update v8-source-location.patch
* chore: fix clippy
* chore: cleanup armhf server ci config
* fix: broken lock file
* fix: c++ version requirement for sysroot builds
* fix: msvc compilation of native modules
* fix: handle rejections for fire-and-forget loadurl
* fix: windows build
* ci: fix teardown of daemon process on windows
```
2026-05-15T20:55:09.7717127Z Assertion failed: !(handle->flags & UV_HANDLE_CLOSING), file src\win\async.c, line 76
```
* chore: update deb and rpm dependencies
* chore: update version info
* spec: improve reliablity of offscreencanvas tests
* spec: retry EPERM failures on teardown
* chore: update x86_64 rpm deps
* ci: exclude server binskim for armhf
* temp: bump distro
* test: ignore deprecation warnings treated as errors
* chore: update lockfile
* fix: externalize electron from auth extension bundles
Error: Cannot find module 'c:\Users\cloudtest\AppData\Local\Temp\vscode-sanityQvCaze\vscode-server-win32-x64-web\extensions\github-authentication\dist\install.js'
at Module._resolveFilename (node:internal/modules/cjs/loader:1476:15)
at wrapResolveFilename (node:internal/modules/cjs/loader:1049:27)
at defaultResolveImplForCJSLoading (node:internal/modules/cjs/loader:1073:10)
at resolveForCJSWithHooks (node:internal/modules/cjs/loader:1094:12)
at Module._load (node:internal/modules/cjs/loader:1262:25)
at wrapModuleLoad (node:internal/modules/cjs/loader:255:19)
at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
at node:internal/main/run_main_module:33:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
Node.js v24.15.0
* test: make wsl sanity tests reliable
* chore: bump electron@42.1.0
* temp: bump distro
* chore: bump electron@42.2.0
* chore: bump distro
* chore: update debian dependencies
* Revert "test: make wsl sanity tests reliable"
This reverts commit b3f2b63e83.
* test: do not fail for deprecation warnings
* chore: patch node24 server binary for wsl1
* chore: address review feedback
* chore: revert global navigation error handler in browserview
* chore: bump distro
VS Code Release Sanity Check Tests
Overview
Automated end-to-end release sanity tests for published VS Code builds. These tests verify critical functionality across different platforms and installation methods, ensuring that published builds meet quality standards before reaching end users.
See Sanity Check wiki page for more details on sanity testing.
Usage
Many tests will use the underlying platform to install and verify basic VS Code functionality. Such tests will need to be run on the corresponding target OS/virtual machine and will fail if ran outside. Use -g or -f command-line options to filter tests to match the host platform.
Command-Line Options
| Option | Alias | Description |
|---|---|---|
--commit <commit> |
-c |
The commit to test (required) |
--quality <quality> |
-q |
The quality to test (required, "stable", "insider" or "exploration") |
--no-cleanup |
Do not cleanup downloaded files after each test | |
--no-signing-check |
Skip Authenticode and codesign signature checks | |
--no-headless |
Run tests with a visible UI (desktop tests only) | |
--no-detection |
Enable all tests regardless of platform and skip executable runs | |
--grep <pattern> |
-g |
Only run tests matching the given pattern (Mocha grep) |
--fgrep <string> |
-f |
Only run tests containing the given string (Mocha fgrep) |
--test-results <path> |
-t |
Output test results in JUnit format to the specified path |
--timeout <sec> |
Set the test-case timeout in seconds (default: 600 seconds) | |
--verbose |
-v |
Enable verbose logging |
--help |
-h |
Show this help message |
Example
To run CLI tests for all platforms on given commit of Insiders build, from the root directory run:
npm run sanity-test -- --commit 19228f26df517fecbfda96c20956f7c521e072be --quality insider -g "cli*"
Scripts
Platform-specific scripts are provided in the scripts/ directory to set up the environment and run tests:
| Script | Platform | Description |
|---|---|---|
run-win32.cmd |
Windows | Runs tests using Edge as the Playwright browser |
run-macOS.sh |
macOS | Installs Playwright WebKit and runs tests |
run-ubuntu.sh |
Ubuntu | Sets up X11, Chromium, and Snap daemon, then runs tests |
run-docker.sh |
Linux (Docker) | Builds and runs tests inside a Docker container |
run-docker.cmd |
Windows (Docker) | Windows wrapper for Docker-based Linux tests |
Docker Script Options
The run-docker.sh script accepts the following options:
| Option | Description |
|---|---|
--container <name> |
Container dockerfile name (required, e.g., "ubuntu", "alpine") |
--arch <arch> |
Target architecture: amd64, arm64, or arm (default: amd64) |
--base-image <image> |
Override the base Docker image (e.g., "ubuntu:24.04") |
All other arguments are passed through to the sanity test runner.
Containers
Docker container definitions are provided in the containers/ directory for testing on various Linux distributions:
| Container | Base Image | Description |
|---|---|---|
alpine |
Alpine 3.x | Alpine Linux with musl libc |
centos |
CentOS Stream 9 | RHEL-compatible distribution |
debian-10 |
Debian 10 (Buster) | Older Debian with legacy library versions |
debian-12 |
Debian 12 (Bookworm) | Current Debian stable |
fedora |
Fedora 36/40 | Cutting-edge RPM-based distribution |
opensuse |
openSUSE Leap 16.0 | SUSE-based enterprise distribution |
redhat |
Red Hat UBI 9 | Red Hat Universal Base Image |
ubuntu |
Ubuntu 22.04/24.04 | Popular Debian-based distribution |
Each container includes:
- Node.js 22.x runtime
- X11 server (Xvfb) for headless desktop testing
- D-Bus for desktop integration
- Architecture-specific VS Code dependencies
Some containers include web browser used for validating web server targets.
Running Tests in a Container
# Ubuntu 24.04 on amd64
./scripts/run-docker.sh --container ubuntu --base-image ubuntu:24.04 -c <commit> -q insider
# Alpine on arm64
./scripts/run-docker.sh --container alpine --arch arm64 -c <commit> -q stable
CI/CD Pipeline
Sanity tests run in Azure Pipelines via the product-sanity-tests.yml pipeline.
Pipeline Parameters
| Parameter | Description |
|---|---|
buildQuality |
The quality of the build to test: "exploration", "insider", or "stable" |
buildCommit |
The published build commit SHA |
npmRegistry |
Custom NPM registry URL (optional) |
Test Matrix
The pipeline tests across multiple platforms and architectures:
Native Hosts:
- macOS arm64
- Windows x64
- Windows arm64
- Ubuntu 22.04 x64 (native, with Snap support)
Partial Support:
For the following platforms only downloads are validated (and not install/runtime):
- macOS x64
Linux Containers (amd64 and arm64):
- Alpine 3.23
- CentOS Stream 9
- Debian 10 and 12 (also arm32)
- Fedora 36 and 40
- openSUSE Leap 16.0
- Red Hat UBI 9
- Ubuntu 22.04 and 24.04 (also arm32)
Pipeline Files
- product-sanity-tests.yml - Main pipeline definition
- sanity-tests.yml - Reusable job template
References
The following public documentation pages provide details on end-user VS Code setup scenarios.