Commit Graph

111 Commits

Author SHA1 Message Date
João Moreno
d2f1f7ce97 Inline Compile & CompileCLI stages into platform specific jobs (#296006)
* chore: ralph loop checkpoint - 2026-02-17 12:48

* fix shell script

* Inline compile stage into platform jobs

Remove the standalone Compile stage from the ADO pipeline. Each
platform job (Windows, Linux, macOS, Alpine, Web) now compiles
TypeScript itself instead of downloading a shared Compilation artifact.

- Add VSCODE_RUN_CHECKS parameter to Linux jobs for hygiene/lint/CG
- Add VSCODE_RUN_COMPILE_EXTRAS parameter to macOS jobs for telemetry
  extraction and sourcemap upload
- Remove VSCODE_COMPILE_ONLY parameter entirely
- Delete product-compile.yml (no longer referenced)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Always run telemetry extraction on macOS builds

The macOS Universal app merge requires both x64 and arm64 builds to have
identical file sets. Telemetry extraction was only running on arm64 (via
VSCODE_RUN_COMPILE_EXTRAS), causing the universal merge to fail due to
missing telemetry-core.json and telemetry-extensions.json in the x64 build.

Move telemetry extraction outside the VSCODE_RUN_COMPILE_EXTRAS gate so
it runs on all macOS builds. Sourcemap upload remains gated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Run telemetry extraction on all client builds

All client builds (Linux, Windows, macOS) need telemetry-core.json and
telemetry-extensions.json. Previously only macOS arm64 ran extract-telemetry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Only run telemetry extraction on Linux x64 (client build)

Linux arm64 and armhf are server-only builds, no need for telemetry files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove .ralph scaffolding files

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix telemetry extraction on Windows with native PowerShell

The bash extract-telemetry.sh script fails on Windows because the Unix
bin shim gets interpreted as Node.js code. Use a native PowerShell
implementation that calls the extractor JS entry point directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Handle missing telemetry files gracefully on Windows

The telemetry extractor may skip emitting declarations-resolved.json
when no events are found. Handle this case with a warning instead of
failing the build.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use deterministic build date from git commit timestamp

When each platform compiles independently, the build date embedded in
cli.js (via INSERT_PRODUCT_CONFIGURATION) differs between machines
because each uses new Date().toISOString(). This causes the macOS
Universal app merge to fail since cli.js SHA differs between x64/arm64.

Fix: use the git commit date (git log -1 --format=%cI HEAD) instead
of the current wall-clock time. This ensures all independent builds
on different machines produce identical timestamps.

Updated in:
- build/lib/date.ts: writeISODate() uses git commit date
- build/next/index.ts: bundle(), transpile, and readISODate fallback
  all use git commit date

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant extensions-ci task from CI pipelines

core-ci (esbuild path) already includes cleanExtensionsBuildTask,
compileNonNativeExtensionsBuildTask, and compileExtensionMediaBuildTask.
Running extensions-ci in parallel caused a race condition where
core-ci's rimraf of .build/extensions clashed with extensions-ci
writing to the same directory.

Also removes dead code:
- extensions-ci and extensions-ci-pr task definitions (fully subsumed)
- core-ci-old task (useEsbuildTranspile is always true)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove VSCODE_RUN_COMPILE_EXTRAS

* address PR feedback regarding code duplication of `getGitCommitDate()` function by exporting a single helper function from `build/lib/date.ts` and importing it in `build/next/index.ts` to ensure consistent behavior and improve code maintainability.

* update readISODate function to return git commit date instead of current date

* add telemetry sorting script and integrate into build process for consistent output

* refactor telemetry extraction process: replace shell script with TypeScript implementation

* update skill

* update telemetry-extractor dependency to version 1.19.0

* fix build

* fix more duplicate telemetry definition issues

* cleanup

* refactor: consolidate validation checks into quality checks and remove obsolete tasks

* bust the cache

* undo cache bust

* fix expression

* fix

* fix: update Azure storage account name in quality checks

* fix: initialize stages set with 'Quality'

* fix: add VSCODE_BUILD_TYPE parameter with options for Product and CI builds

* fix: update Azure Pipeline CLI to use parameters instead of variables for queueing builds

* fix: update VSCODE_BUILD_TYPE parameter values for clarity

* fix: update default value for VSCODE_BUILD_TYPE parameter to 'Product'

* leaner

* even leaner

* only npm ci in build

* 💄

* run entire npm ci

* fix

* fix

* fix it

* Inline CompileCLI into platform stages

- Remove centralized CompileCLI stage
- Move CLI jobs into Windows, Linux, macOS stages as independent jobs
- CLI jobs now compile, publish unsigned mid-job, sign, and publish signed
- Platform compile jobs use deemon + waitForArtifacts for async CLI download
- Delete separate CLI sign jobs (now merged into CLI compile jobs)
- Remove CompileCLI from publish.ts stage tracking

* fix: macOS CLI signing - use proper directory structure for ESRP

* fix: add BUILDS_API_URL to Windows and Linux job templates

* fix: label Linux CLI jobs clearly

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: João Moreno <22350+joaomoreno@users.noreply.github.com>
2026-03-01 08:48:08 +01:00
Osvaldo Ortega
e166b7b679 Revert "Revert "debt - remove AMD support of VSCode for web"" (#286423)
Revert "Revert "debt - remove AMD support of VSCode for web" (#286421)"

This reverts commit 84ce6e1a98.
2026-01-07 13:02:55 -08:00
Osvaldo Ortega
84ce6e1a98 Revert "debt - remove AMD support of VSCode for web" (#286421)
Revert "debt - remove AMD support of VSCode for web (#285230)"

This reverts commit 74c4ecddf7.
2026-01-07 19:30:20 +00:00
Benjamin Pasero
74c4ecddf7 debt - remove AMD support of VSCode for web (#285230) 2025-12-28 20:12:52 +01:00
Matt Bierner
f0e7fe3775 Convert a few more build scripts to TypeScript
For #277526
2025-11-21 09:26:54 -08:00
Matt Bierner
2648263d3e Run our build scripts directly as typescript (#277567)
* Run our build scripts directly as typescript #277567

Follow up on #276864
For #277526

* Remove a few more ts-node references

* Fix linux and script reference

* Remove `_build-script` ref

* Fix script missing closing quote

* use type only import

* Fix export

* Make sure to run copy-policy-dto

* Make sure we run the copy-policy-dto script

* Enable `verbatimModuleSyntax`

* Pipelines fixes

* Try adding explicit ext to path

* Fix bad edit

* Revert extra `--`

---------

Co-authored-by: João Moreno <joaomoreno@users.noreply.github.com>
2025-11-21 14:56:00 +01:00
João Moreno
911a41f96e Revert "Insider builds should have an auto increasing version number" (#278356)
Revert "Insider builds should have an auto increasing version number (#277497)"

This reverts commit 222fb55dd5.
2025-11-19 15:08:11 +01:00
João Moreno
222fb55dd5 Insider builds should have an auto increasing version number (#277497)
* wip

* more logs

* add bump-insiders-version.yml

* ok give this a try

* fix paths

* missing parameter

* ok

* fix

* fix CI

* hm

* rm

* fix appx version

* fix inno

* whoops

* update insiders version in compile job as well

* thanks @lszomoru
2025-11-17 17:21:33 +01:00
Raymond Zhao
ea01b7e2bb chore: fix SBOM names (#274460) 2025-11-02 14:35:48 +01:00
Ladislau Szomoru
4b8bd3cb6f Engineering - refactor product pipeline (#269979) 2025-10-06 11:12:40 +02:00
Ladislau Szomoru
034e8a4bf4 Engineering - add node_modules stage to keep node_modules cache warm (#268839)
* Engineering - scaffold Windows jobs to maintain node_modules cache

* Add agent pool

* No need to extract node_modules

* Delete commented out code

* Add Linux jobs

* Add macOS job

* Add missing condition

* Add Web job

* Remove steps to download compilation output

* Fix condition for the node_modules stage

* Remove parameter

* Add Alpine jobs

* Remove the check to test the stage

* Switch macOS job to x64

* Restore the check for the node_modules stage

* Remove extraction
2025-09-29 13:55:31 +02:00
Ladislau Szomoru
bb58370bb2 Engineering - tweak repository checkout task (#267213)
* Engineering - limit the checkout for the publish and release stages

* Tweak checkout for each job

* Fix path
2025-09-17 16:30:05 +02:00
João Moreno
8d7e69612d revert back to official nodejs mirror (#265692)
maybe the issue is gone by now 🤞:
- https://github.com/microsoft/azure-pipelines-tasks/issues/18280
- https://github.com/nodejs/nodejs.org/issues/4495
2025-09-08 17:14:10 +00:00
João Moreno
0ec8ac6438 Check for artifact existence before publishing (#263777)
* Refactor artifact publishing steps to use a common template across all build configurations

* Refactor conditions in publish artifact template

* typo

* whops

* fix msg

* hm
2025-08-29 15:49:34 +02:00
Robo
35173c1de1 Revert "ci: always generate artifact prefix (#263738)" (#263764)
This reverts commit 502526da29.
2025-08-28 08:31:42 +00:00
Robo
502526da29 ci: always generate artifact prefix (#263738) 2025-08-28 07:22:41 +02:00
Ladislau Szomoru
e09d79667b Engineering - switch over to the new build agents (#244964)
* Engineering - switch CLI stage agents

* Update the macOS stage

* Comment out policy generation to unblock

* ensure node modules cache takes agent's arch into account

---------

Co-authored-by: João Moreno <joaomoreno@users.noreply.github.com>
2025-03-31 16:56:23 +02:00
João Moreno
a46559966e move to new service connection, remove deprecated storage account upload (#233191) 2024-11-06 14:57:38 +01:00
João Moreno
1f8fd7adef use oidc (#233126)
* use oidc

* undo dep bump

* use ClientAssertionCredential

* temporarily disable mangle

* adopt oidc
2024-11-05 21:50:01 +01:00
Benjamin Pasero
9bd60d090a esm - better content mapper to address sourcemaps (#230476) 2024-10-04 15:34:21 +02:00
Robo
8422282f04 chore: update to Electron 32 (#218245)
* chore: update electron@31.0.1

* chore: bump electron@31.1.0

* chore: update sysroot download url

* chore: bump electron@31.3.0

* chore: update rpm deps for arm64

* chore: bump electron@31.4.0

* chore: bump electron@32.0.0

* fix: compile stage

* fix: install on windows

* fix: override node-addon-api to support c++20 with gcc-11

Brings in fed13534c5
to support compilation with gcc>=11

* ci: use gnu++2a for gcc <=9

* ci: fix sdl pipeline

* ci: web stage

* ci: fix alpine stage

* ci: fix legacy server stage

* fix: node-addon-api resolution for remote

* ci: fix legacy arm stage

* chore: sync npm dependencies

* chore: bump electron@32.0.1

* ci: fix monaco editor checks

* ci: fix legacy server stage

* ci: possible concurrent execution

* ci: use foreground scripts on windows

* chore: bump electron@32.0.2

* fix: compilation

Refs 204011c3f2

* Revert "wco - hardcode devtools location on Linux (#227084)"

This reverts commit dfb96d11d3.

* chore: bump electron@32.1.2

* ci: remove workaround for missing c++20 flags

* ci: use gnu++2a for gcc <=9

* chore: bump distro

* ci: use gnu++17 for legacy arm servers

* fixup! override node-addon-api to support c++20 with gcc-11

Scope to only the affected package versions

* chore: bump @vscode/policy-watcher

* chore: bump @vscode/sqlite3

* chore: bump node.js v20.17.0

* chore: bump distro

* chore: update @vscode/windows-ca-certs

* chore: update builds for electron@32.1.2

* chore: bump distro

* chore: bump dependencies

* ci: fix arm legacy servers
2024-10-04 19:19:06 +09:00
Robo
a287cd4260 ci: fix paths to apt retry script (#230274) 2024-10-02 02:35:33 +02:00
Robo
9d3abdc8ff ci: apply retry logic for apt failures (#230217) 2024-10-01 13:01:10 +02:00
Robo
b5a6aa14a8 feat: switch to npm as default package manager (#226927)
* feat: move from yarn to npm

* chore: skip yarn.lock files

* fix: playwright download

* chore: fix compile and hygiene

* chore: bump vsce@2.17.0

Refs 8b49e9dfdf

* test: update results for bat and sh colorizer tests

* fix: add missing lock files for windows

* fix: switch to legacy-peer-deps

* chore: update markdown-it@14.1.0

Refs 737c95a129

esbuild step in extensions-ci-pr was previously using markdown-it
from root which had userland punycode and was able to compile successfully.

* ci: increase pr timeout for windows integration tests

* chore: fix product build

* build: ignore extension dev dependency for rcedit

* build: fix working directory inside container

* build: fix dependency generation

* npm: update dependencies

* ci: use global npmrc

* ci: update cache

* ci: setup global npmrc for private npm auth

* build: fix extension bundling

* chore: sync npm dependencies

* ci: debug env variables for container

* ci: fix win32 cli pipeline

* build: fix npmrc config usage for build/ and remote/ dirs

* fix: windows build

* fix: container builds

* fix: markdown-language-features tests and bundling

```
[03:58:22] Error: Command failed: /Users/demohan/.nvm/versions/node/v20.15.1/bin/node /Users/demohan/github/vscode/extensions/markdown-language-features/esbuild-notebook.js --outputRoot /Users/demohan/github/vscode/.build/extensions/markdown-language-features
✘ [ERROR] Could not resolve "punycode"

    extensions/markdown-language-features/node_modules/markdown-it/lib/index.js:14:27:
      14 │ var punycode     = require('punycode');
         ╵                            ~~~~~~~~~~

  The package "punycode" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.
```

Adds userland package based on beed9aee2c

* fix: container builds for distro

* chore: update yarn occurrences

* fixup! chore: bump vsce@2.17.0

Uses the closest version to `main` branch that does not
include d3cc84cdec
while still having the fix 8b49e9dfdf

* chore: sync npm dependencies

* chore: sync npm dependencies

* chore: sync npm dependencies

* chore: throw error when yarn is used for installation

* chore: add review feedback

* chore: switch exec => run where needed

* chore: npm sync dependencies

* fix: markdown-language-features bundling

```
✘ [ERROR] Could not resolve "punycode"

    extensions/markdown-language-features/node_modules/markdown-it/lib/index.js:14:27:
      14 │ var punycode     = require('punycode');
         ╵                            ~~~~~~~~~~

  The package "punycode" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.
```

Adds missing userland package based on markdown-it/markdown-it@beed9ae,
can be removed once we update markdown-it >= 14.1.0

* ci: rename no-yarn-lock-changes.yml

* chore: sync npm dependencies

* ci: restore no-yarn-lock-changes.yml

We can disable it in a separate PR to keep the required
checks happy and also need workflow edit perms.

* chore: sync npm dependencies

* ci: rebuild cache

* ci: fix no-package-lock-changes.yml

* chore: bump distro

* chore: rm yarn.lock files

* chore: rm yarn.lock files without dependencies

* chore: add vscode-selfhost-import-aid to postinstall dirs

* chore: bump distro
2024-09-06 22:18:02 +09:00
Kevin Cui
d1eab2bcf4 ci: ensure retry logic consistency (#226038)
Related PR: #171489

Signed-off-by: Kevin Cui <bh@bugs.cc>
2024-09-04 04:51:19 +00:00
Ladislau Szomoru
310f0a3d85 Engineering - upgrade AzureKeyVault task (#227191) 2024-08-30 11:10:44 +02:00
Benjamin Pasero
f6f90e0163 Implement NLS without AMD loader (#214588) 2024-06-28 11:55:48 +02:00
João Moreno
addd445017 upload sourcemaps to CDN instead of ticino (#214141) 2024-06-03 16:10:01 +02:00
Robo
5216c04428 chore: update to electron 29 (#209818)
* chore: update electron@29.1.0

* chore: update typings to 20.x

* chore: bump electron@29.1.5

* ci: fix crash in compiling extensions-ci

* chore: disable .d.ts check for build/

$ ../node_modules/.bin/tsc -p tsconfig.build.json
node_modules/@types/chokidar/index.d.ts:21:14 - error TS2420: Class 'import("/Users/demohan/github/vscode/build/node_modules/@types/chokidar/index").FSWatcher' incorrectly implements interface 'import("fs").FSWatcher'.
  Type 'FSWatcher' is missing the following properties from type 'FSWatcher': ref, unref

21 export class FSWatcher extends EventEmitter implements fs.FSWatcher {
                ~~~~~~~~~

node_modules/chokidar/types/index.d.ts:8:14 - error TS2420: Class 'import("/Users/demohan/github/vscode/build/node_modules/chokidar/types/index").FSWatcher' incorrectly implements interface 'import("fs").FSWatcher'.
  Type 'FSWatcher' is missing the following properties from type 'FSWatcher': ref, unref

8 export class FSWatcher extends EventEmitter implements fs.FSWatcher {
               ~~~~~~~~~

Found 2 errors in 2 files.

Errors  Files
     1  node_modules/@types/chokidar/index.d.ts:21
     1  node_modules/chokidar/types/index.d.ts:8

Refs a0f9e09f64

* chore: update core types

* temp: fix layer validation

* chore: update nodejs checksums

* ci: use latest v20 LTS for missing node-gyp

Refs eacec5f490

* ci: define LIBCPP_HARDENING_MODE

* ci: fix crash in vscode-web-min-ci

* chore: update rpm deps-list

* chore: bump tree-sitter-typescript@0.20.5

* chore: bump electron@29.3.0

* chore: bump electron@29.3.1

* chore: update rpm deps-list for x86_64

* ci: disable io_uring UV backend on linux

* ci: disable io_uring backend for oss as well

* chore: update typings to 20.x

* ci: add TODO for io_uring workaround

* chore: bump distro

* chore: update preinstall node version checks

* chore: update @types/gulp

Refs https://github.com/microsoft/vscode/issues/212442

* ci: disable io_uring in more test suites
2024-05-11 01:20:28 +09:00
João Moreno
e5515ac702 Adopt 1ES PT (#207844)
* extend 1es pipeline template

* oops

* fix template references

* argh

* hmm

* hm

* hm

* use outputs for compilation artifact

* this

* use 1ES.PublishPipelineArtifact@1 instead of publish

* more 1ES.PublishPipelineArtifact@1 adoption

* provide windows pool for sdl sources

* sdl

* fix pools

* fix macos

* disable sbom for intermediate artifacts

* use mariner linux

* try inline tsa options

* fix credscan

* hm

* sudo it

* more suppressions

* be explicit with SBOM build drop paths

* fix indentation

* fix file extensions

* fix cli sbom build drop paths

* fix more build

* fix unzip cli

* careful with _manifest in artifacts

* do not close file

* add logging

* debug

* use snapcraft container

* remove size check

* fix macos cli step

* fix snap permissions

* fix macos

* better logs

* fix snap

* more cred scan suppressions

* even more supressiong

* alpine-arm64: try using x64

* Revert "alpine-arm64: try using x64"

This reverts commit bf2003bf60.

* test docker

* I wonder

* logs

* hm

* fix indentation

* hm

* hm

* fix snap finds

* remove auth

* use hostArchitecture

* snap: limit find

* hm

* sudo

* Add validateToolOutput: None to the build pipeline

* bring back sdl-scan

* try all tools: true

* use release

* Update Azure Pipelines YAML file for Linux product build and test

* hm

* hm

* same for win32

* hm

* hm

* Revert "hm"

This reverts commit 1b9dcae85b.

* use branch

* fix template file

* fix template paths
2024-03-21 14:16:32 +01:00
João Moreno
417e17038c Use NodeJS mirror for more reliable downloads (#196038)
get node from mirror
2023-10-20 09:53:57 +02:00
Raymond Zhao
880cb517e1 Add more SBOMs (#195736) 2023-10-17 15:55:55 -07:00
Rob Lourens
7d4027b62b Test using nvmrc in CI (#194836)
* Test using nvmrc in CI

* Force run yarn

* fix

* Bump nvmrc to latest

* Revert "Force run yarn"

This reverts commit af741f37d5.

* Make all jobs use node from nvmrc
And install node-gyp where needed
2023-10-04 19:08:50 -07:00
Rob Lourens
8ee50f2e21 Fix the rest of the node version references (#194677) 2023-10-03 00:55:32 +01:00
Robo
ea490e5545 chore: update to Electron 25 (#188268)
* chore: update electron@25.3.0

* ci: update NodeTool version

* chore: update @types/node

* add more common types to layers checker

* chore: update debian dependencies

* chore: update rpm dependencies

* fix: use legacy dns result order of Node.js

* ci: remove deprecated always-auth npm config

Refs npm/cli@72a7eeb

* chore: update deb and rpm dependencies

* chore: update armhf rpm dependencies

* chore: update x64 debian dependencies

* chore: update x64 rpm dependencies

* chore: update electron@25.3.1

* chore: update electron@25.4.0

* chore: bump distro

* chore: bump distro

---------

Co-authored-by: Benjamin Pasero <benjamin.pasero@microsoft.com>
2023-08-07 21:46:27 +09:00
Robo
7628b550c0 ci: fix oss node_modules cache pipeline (#188448)
* ci: fix oss node_modules cache pipeline

* align all "Install build dependencies" steps

---------

Co-authored-by: João Moreno <joao.moreno@microsoft.com>
2023-07-21 09:40:53 +02:00
Christof Marti
6ae441b56f Kerberos auth for proxies (#188130) 2023-07-19 20:11:00 +02:00
João Moreno
6d1066875e isolate artifacts from attempted runs (#187437)
* isolate artifacts from attempted runs

* set ARTIFACT_PREFIX to empty string first

* cleaner
2023-07-10 16:15:29 +02:00
João Moreno
0917209e9a Always publish build artifacts, if built (#187134)
* always publish build artifacts, if built

fixes #187122

* fix windows build

* fix builds

* fix

* more fixes

* further debug

* fix yarn

* remove debug

* remove 7zip dependency

* fix snap
2023-07-07 11:45:35 -07:00
João Moreno
14ea89e7ca build: quieter node module cache logging (#180709) 2023-04-25 08:26:56 +02:00
João Moreno
aac80c3f0a Adopt new distro format (#178689)
* Simplify distro

Also a ton of drive-by fixing around builds:
- simplified many oneliners
- fixed missing custom npm registry call setups
- remove unnecessary and duplicate work during builds
- many many fixes
2023-04-02 09:35:39 +02:00
João Moreno
e8fa61d2fc retry terrapin 5 times (#171489)
fixes #170935
2023-01-17 13:48:02 +01:00
João Moreno
59814267dd Use new Key Vault account (#166049)
use separate keyvault
2022-11-11 09:38:36 +01:00
Alexandru Dima
ba91a81c16 Always patch the npm registry in yarn.lock files and compute the node modules cache key after having patched them (#164795) 2022-10-27 12:31:15 +02:00
João Moreno
8fe6f3e917 Allow custom npm registries at build time (#163629)
* use azure artifacts as new terrapin solution

* debug

* skip npm auth

* bring back

* also configure yarn

* custom npm registry

* oops

* powershell

* use none instead of empty string

* uff
2022-10-24 09:24:44 +02:00
João Moreno
ac9aff3521 reenable terrapin (#163622)
* drop npx for terrapin

* enable terrapin by default
2022-10-14 01:35:15 -07:00
Connor Peet
fbda011715 build: do proper hashing for built-in dependencies (#157295)
Fixes #157244
2022-08-05 13:00:33 -07:00
Connor Peet
c9df538b06 build: apply built-in ext cache to the rest of the pipeline (#156939) 2022-08-02 16:26:17 -07:00
Ladislau Szomoru
2be99473d5 Engineering - execute tests in parallel for the CI build (#151593) 2022-06-10 04:35:28 -07:00
João Moreno
c5902c4e2b hm 2022-03-21 14:46:14 +01:00