* feat: move from yarn to npm * chore: skip yarn.lock files * fix: playwright download * chore: fix compile and hygiene * chore: bump vsce@2.17.0 Refs8b49e9dfdf* test: update results for bat and sh colorizer tests * fix: add missing lock files for windows * fix: switch to legacy-peer-deps * chore: update markdown-it@14.1.0 Refs737c95a129esbuild step in extensions-ci-pr was previously using markdown-it from root which had userland punycode and was able to compile successfully. * ci: increase pr timeout for windows integration tests * chore: fix product build * build: ignore extension dev dependency for rcedit * build: fix working directory inside container * build: fix dependency generation * npm: update dependencies * ci: use global npmrc * ci: update cache * ci: setup global npmrc for private npm auth * build: fix extension bundling * chore: sync npm dependencies * ci: debug env variables for container * ci: fix win32 cli pipeline * build: fix npmrc config usage for build/ and remote/ dirs * fix: windows build * fix: container builds * fix: markdown-language-features tests and bundling ``` [03:58:22] Error: Command failed: /Users/demohan/.nvm/versions/node/v20.15.1/bin/node /Users/demohan/github/vscode/extensions/markdown-language-features/esbuild-notebook.js --outputRoot /Users/demohan/github/vscode/.build/extensions/markdown-language-features ✘ [ERROR] Could not resolve "punycode" extensions/markdown-language-features/node_modules/markdown-it/lib/index.js:14:27: 14 │ var punycode = require('punycode'); ╵ ~~~~~~~~~~ The package "punycode" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error. ``` Adds userland package based onbeed9aee2c* fix: container builds for distro * chore: update yarn occurrences * fixup! chore: bump vsce@2.17.0 Uses the closest version to `main` branch that does not included3cc84cdecwhile still having the fix8b49e9dfdf* chore: sync npm dependencies * chore: sync npm dependencies * chore: sync npm dependencies * chore: throw error when yarn is used for installation * chore: add review feedback * chore: switch exec => run where needed * chore: npm sync dependencies * fix: markdown-language-features bundling ``` ✘ [ERROR] Could not resolve "punycode" extensions/markdown-language-features/node_modules/markdown-it/lib/index.js:14:27: 14 │ var punycode = require('punycode'); ╵ ~~~~~~~~~~ The package "punycode" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error. ``` Adds missing userland package based on markdown-it/markdown-it@beed9ae, can be removed once we update markdown-it >= 14.1.0 * ci: rename no-yarn-lock-changes.yml * chore: sync npm dependencies * ci: restore no-yarn-lock-changes.yml We can disable it in a separate PR to keep the required checks happy and also need workflow edit perms. * chore: sync npm dependencies * ci: rebuild cache * ci: fix no-package-lock-changes.yml * chore: bump distro * chore: rm yarn.lock files * chore: rm yarn.lock files without dependencies * chore: add vscode-selfhost-import-aid to postinstall dirs * chore: bump distro
4.3 KiB
VS Code Smoke Test
Make sure you are on Node v12.x.
Quick Overview
# Build extensions in the VS Code repo (if needed)
npm i && npm run compile
# Dev (Electron)
npm run smoketest
# Dev (Web - Must be run on distro)
npm run smoketest -- --web --browser [chromium|webkit]
# Build (Electron)
npm run smoketest -- --build <path to latest version>
example: npm run smoketest -- --build /Applications/Visual\ Studio\ Code\ -\ Insiders.app
# Build (Web - read instructions below)
npm run smoketest -- --build <path to server web build (ends in -web)> --web --browser [chromium|webkit]
# Remote (Electron)
npm run smoketest -- --build <path to latest version> --remote
* This step is necessary only when running without --build and OSS doesn't already exist in the .build/electron directory.
Running for a release (Endgame)
You must always run the smoketest version that matches the release you are testing. So, if you want to run the smoketest for a release build (e.g. release/1.22), you need to check out that version of the smoke tests too:
git fetch
git checkout release/1.22
npm i && npm run compile
cd test/smoke
npm i
Web
There is no support for testing an old version to a new one yet.
Instead, simply configure the --build command line argument to point to the absolute path of the extracted server web build folder (e.g. <rest of path here>/vscode-server-darwin-x64-web for macOS). The server web build is available from the builds page (see previous subsection).
macOS: if you have downloaded the server with web bits, make sure to run the following command before unzipping it to avoid security issues on startup:
xattr -d com.apple.quarantine <path to server with web folder zip>
Note: make sure to point to the server that includes the client bits!
Debug
--verboselogs all the low level driver calls made to Code;-f PATTERN(alias-g PATTERN) filters the tests to be run. You can also use pretty much any mocha argument;--headlesswill run playwright in headless mode when--webis used.
Note: you can enable verbose logging of playwright library by setting a DEBUG environment variable before running the tests (https://playwright.dev/docs/debug#verbose-api-logs), for example to pw:browser.
Develop
cd test/smoke
npm run watch
Troubleshooting
Error: Could not get a unique tmp filename, max tries reached
On Windows, check for the folder C:\Users\<username>\AppData\Local\Temp\t. If this folder exists, the tmp module can't run properly, resulting in the error above. In this case, delete the t folder.
Pitfalls
-
Beware of workbench state. The tests within a single suite will share the same state.
-
Beware of singletons. This evil can, and will, manifest itself under the form of FS paths, TCP ports, IPC handles. Whenever writing a test, or setting up more smoke test architecture, make sure it can run simultaneously with any other tests and even itself. All test suites should be able to run many times in parallel.
-
Beware of focus. Never depend on DOM elements having focus using
.focusedclasses or:focuspseudo-classes, since they will lose that state as soon as another window appears on top of the running VS Code window. A safe approach which avoids this problem is to use thewaitForActiveElementAPI. Many tests use this whenever they need to wait for a specific element to have focus. -
Beware of timing. You need to read from or write to the DOM... but is it the right time to do that? Can you 100% guarantee that
inputbox will be visible at that point in time? Or are you just hoping that it will be so? Hope is your worst enemy in UI tests. Example: just because you triggered Quick Access withF1, it doesn't mean that it's open and you can just start typing; you must first wait for the input element to be in the DOM as well as be the current active element. -
Beware of waiting. Never wait longer than a couple of seconds for anything, unless it's justified. Think of it as a human using Code. Would a human take 10 minutes to run through the Search viewlet smoke test? Then, the computer should even be faster. Don't use
setTimeoutjust because. Think about what you should wait for in the DOM to be ready and wait for that instead.