Commit Graph

291 Commits

Author SHA1 Message Date
João Moreno
d2f1f7ce97 Inline Compile & CompileCLI stages into platform specific jobs (#296006)
* chore: ralph loop checkpoint - 2026-02-17 12:48

* fix shell script

* Inline compile stage into platform jobs

Remove the standalone Compile stage from the ADO pipeline. Each
platform job (Windows, Linux, macOS, Alpine, Web) now compiles
TypeScript itself instead of downloading a shared Compilation artifact.

- Add VSCODE_RUN_CHECKS parameter to Linux jobs for hygiene/lint/CG
- Add VSCODE_RUN_COMPILE_EXTRAS parameter to macOS jobs for telemetry
  extraction and sourcemap upload
- Remove VSCODE_COMPILE_ONLY parameter entirely
- Delete product-compile.yml (no longer referenced)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Always run telemetry extraction on macOS builds

The macOS Universal app merge requires both x64 and arm64 builds to have
identical file sets. Telemetry extraction was only running on arm64 (via
VSCODE_RUN_COMPILE_EXTRAS), causing the universal merge to fail due to
missing telemetry-core.json and telemetry-extensions.json in the x64 build.

Move telemetry extraction outside the VSCODE_RUN_COMPILE_EXTRAS gate so
it runs on all macOS builds. Sourcemap upload remains gated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Run telemetry extraction on all client builds

All client builds (Linux, Windows, macOS) need telemetry-core.json and
telemetry-extensions.json. Previously only macOS arm64 ran extract-telemetry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Only run telemetry extraction on Linux x64 (client build)

Linux arm64 and armhf are server-only builds, no need for telemetry files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove .ralph scaffolding files

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix telemetry extraction on Windows with native PowerShell

The bash extract-telemetry.sh script fails on Windows because the Unix
bin shim gets interpreted as Node.js code. Use a native PowerShell
implementation that calls the extractor JS entry point directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Handle missing telemetry files gracefully on Windows

The telemetry extractor may skip emitting declarations-resolved.json
when no events are found. Handle this case with a warning instead of
failing the build.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use deterministic build date from git commit timestamp

When each platform compiles independently, the build date embedded in
cli.js (via INSERT_PRODUCT_CONFIGURATION) differs between machines
because each uses new Date().toISOString(). This causes the macOS
Universal app merge to fail since cli.js SHA differs between x64/arm64.

Fix: use the git commit date (git log -1 --format=%cI HEAD) instead
of the current wall-clock time. This ensures all independent builds
on different machines produce identical timestamps.

Updated in:
- build/lib/date.ts: writeISODate() uses git commit date
- build/next/index.ts: bundle(), transpile, and readISODate fallback
  all use git commit date

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant extensions-ci task from CI pipelines

core-ci (esbuild path) already includes cleanExtensionsBuildTask,
compileNonNativeExtensionsBuildTask, and compileExtensionMediaBuildTask.
Running extensions-ci in parallel caused a race condition where
core-ci's rimraf of .build/extensions clashed with extensions-ci
writing to the same directory.

Also removes dead code:
- extensions-ci and extensions-ci-pr task definitions (fully subsumed)
- core-ci-old task (useEsbuildTranspile is always true)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove VSCODE_RUN_COMPILE_EXTRAS

* address PR feedback regarding code duplication of `getGitCommitDate()` function by exporting a single helper function from `build/lib/date.ts` and importing it in `build/next/index.ts` to ensure consistent behavior and improve code maintainability.

* update readISODate function to return git commit date instead of current date

* add telemetry sorting script and integrate into build process for consistent output

* refactor telemetry extraction process: replace shell script with TypeScript implementation

* update skill

* update telemetry-extractor dependency to version 1.19.0

* fix build

* fix more duplicate telemetry definition issues

* cleanup

* refactor: consolidate validation checks into quality checks and remove obsolete tasks

* bust the cache

* undo cache bust

* fix expression

* fix

* fix: update Azure storage account name in quality checks

* fix: initialize stages set with 'Quality'

* fix: add VSCODE_BUILD_TYPE parameter with options for Product and CI builds

* fix: update Azure Pipeline CLI to use parameters instead of variables for queueing builds

* fix: update VSCODE_BUILD_TYPE parameter values for clarity

* fix: update default value for VSCODE_BUILD_TYPE parameter to 'Product'

* leaner

* even leaner

* only npm ci in build

* 💄

* run entire npm ci

* fix

* fix

* fix it

* Inline CompileCLI into platform stages

- Remove centralized CompileCLI stage
- Move CLI jobs into Windows, Linux, macOS stages as independent jobs
- CLI jobs now compile, publish unsigned mid-job, sign, and publish signed
- Platform compile jobs use deemon + waitForArtifacts for async CLI download
- Delete separate CLI sign jobs (now merged into CLI compile jobs)
- Remove CompileCLI from publish.ts stage tracking

* fix: macOS CLI signing - use proper directory structure for ESRP

* fix: add BUILDS_API_URL to Windows and Linux job templates

* fix: label Linux CLI jobs clearly

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: João Moreno <22350+joaomoreno@users.noreply.github.com>
2026-03-01 08:48:08 +01:00
Raymond Zhao
46d0e3c4ba chore: enable BinSkim on Linux (#278183) 2025-11-24 11:08:10 -08:00
Ladislau Szomoru
4b8bd3cb6f Engineering - refactor product pipeline (#269979) 2025-10-06 11:12:40 +02:00
Ladislau Szomoru
8c58788416 Engineering - refactor Linux stage (#268828)
* Engineering - refactor Linux stage

* Declare outputs in the template context

* Add outputParentDirectory

* Tweak ouput directory

* More artifacts

* deb and rpm packages

* Fixing path

* Another attempt to fix the path

* Add back the rpm gulp task

* Enable snap package

* Add template for ci jobs

* Test ci jobs

* Add default values for parameters

* Restore the correct check
2025-09-29 12:43:54 +02:00
Ladislau Szomoru
bb58370bb2 Engineering - tweak repository checkout task (#267213)
* Engineering - limit the checkout for the publish and release stages

* Tweak checkout for each job

* Fix path
2025-09-17 16:30:05 +02:00
João Moreno
8d7e69612d revert back to official nodejs mirror (#265692)
maybe the issue is gone by now 🤞:
- https://github.com/microsoft/azure-pipelines-tasks/issues/18280
- https://github.com/nodejs/nodejs.org/issues/4495
2025-09-08 17:14:10 +00:00
João Moreno
6c5f088d17 allow skipping linux snap (#265311)
* allow skipping linux snap

* fix ci
2025-09-05 10:30:38 +00:00
João Moreno
0ec8ac6438 Check for artifact existence before publishing (#263777)
* Refactor artifact publishing steps to use a common template across all build configurations

* Refactor conditions in publish artifact template

* typo

* whops

* fix msg

* hm
2025-08-29 15:49:34 +02:00
Robo
35173c1de1 Revert "ci: always generate artifact prefix (#263738)" (#263764)
This reverts commit 502526da29.
2025-08-28 08:31:42 +00:00
Robo
502526da29 ci: always generate artifact prefix (#263738) 2025-08-28 07:22:41 +02:00
Ladislau Szomoru
cb596f81ac Engineering - cleanup pull request ADO pipeline (#262276)
* Initial implementation

* Fix indentation
2025-08-19 11:25:14 +02:00
Robo
10575e19aa (partial revert) ci: remove install build dependencies step on linux (#257064) 2025-07-21 11:25:26 +00:00
Robo
b37c013d08 ci: remove install build dependencies step on linux (#257050) 2025-07-21 09:38:54 +00:00
Robo
6947b69d47 fix: revert remote libcxx requirement to 3.4.25 (#248976)
* fix: revert libcxx requirement to 3.4.25

* ci: temp debugging

* fix: use gcc 8.5.0 for remote

* ci: update cache

* ci: fix sysroot directory

* ci: fix objdump directory
2025-05-14 20:10:32 -07:00
Robo
12326f9906 chore: update to Electron 35 (#245423)
* chore: bump electron@35.0.1

* chore: update types/node@22.x

* chore: update web types for node v22.x

* chore: bump electron@35.1.2

* chore: update node.js build

* chore: update app.dock usage

Refs 71f3ff6bf2

* chore: bump electron@35.1.4

* chore: bump electron@35.1.5

* ci: bump sysroot to glibc 2.28 and gcc 10.5.0

* ci: enable timeout for smoketests

* chore: bump min glibcxx to 3.4.26 for server

* Revert "ci: enable timeout for smoketests"

This reverts commit afb637e85d.

* chore: update debian dependencies

* fix: workaround npm.ps1 argument parsing with powershell

* chore: update rpm dependencies

* test: partially revert changes from 242535

* test: remove redudant keybinding dispatch for selectTab

* test: fix test failure from running configured tasks

* test: focus settings editor for preferences.test.ts

* node - adopt compile cache (#246835)

* node - adopt compile cache

* adopt for utility process

* tweaks

* log state of compilation cache

* Revert "log state of compilation cache"

This reverts commit f3840387a583013834762e2c44f6e8424929297f.

* Revert "node - adopt compile cache (#246835)"

This reverts commit 673a00cab66c9bac87f3cd27b80efa41c80150f1.

* chore: update builds

* chore: bump electron@35.2.0

* chore: bump electron@35.2.1

* chore: bump electron@35.2.2

---------

Co-authored-by: Benjamin Pasero <benjamin.pasero@microsoft.com>
2025-05-09 19:03:36 +09:00
Ladislau Szomoru
0bb72be9d0 Engineering - linux codesign in the background (#246344)
* Try things oout

* Add sleep

* Bump deemon

* .

* Improve script

* bump deemon

* Revert some debugging changes

* Revert more code used for debugging

---------

Co-authored-by: João Moreno <joaomoreno@users.noreply.github.com>
2025-04-11 21:31:43 +02:00
João Moreno
77a0b47822 fix linux stage (#246235) 2025-04-10 20:33:31 +02:00
João Moreno
7ffe461fac inline linux snap (#246076)
* inline linux snap

* hm

* no need for tar

* hm

* hm

* debug

* hm

* 💄
2025-04-10 17:24:29 +02:00
Ladislau Szomoru
88b8f12fd4 Engineering - linux codesign in parallel (#246135)
* Set up the jobs and tasks

* Add codesign script

* Debug things

* Speed things up while testing

* Add more logging

* More debugging

* Try to use powershell

* Another try

* Another try

* Remove debugging code

* Remove debugging code

* .

* Try using pwsh

* Fix script

* Maybe now

* Maybe now

* Run the sign tasks in parallel

* Revert debug change
2025-04-09 22:02:25 +02:00
Robo
b952f59bea ci: remove explicit dbus setup (#246114) 2025-04-09 17:29:50 +02:00
Ladislau Szomoru
5f65e5f6e9 Engineering - run tests based on target (#245367) 2025-04-03 06:27:23 +00:00
Ladislau Szomoru
d17c002ece Engineering - PR pipeline improvements (#245331) 2025-04-02 18:35:48 +02:00
Ladislau Szomoru
e09d79667b Engineering - switch over to the new build agents (#244964)
* Engineering - switch CLI stage agents

* Update the macOS stage

* Comment out policy generation to unblock

* ensure node modules cache takes agent's arch into account

---------

Co-authored-by: João Moreno <joaomoreno@users.noreply.github.com>
2025-03-31 16:56:23 +02:00
Robo
9df979cc9a ci: fix installation of build dependencies (#239290)
* ci: fix installation of build dependencies

* ci: add missing quality parameter
2025-01-31 09:27:39 +00:00
João Moreno
dfe78a1fb9 use WIF for Codesign (#233908)
* use WIF for Codesign

* fix indentation

* fix tasks
2024-11-15 22:16:31 +01:00
João Moreno
0244c5edf7 move to new esrp service connection (#233591) 2024-11-11 12:52:05 -08:00
João Moreno
a46559966e move to new service connection, remove deprecated storage account upload (#233191) 2024-11-06 14:57:38 +01:00
Robo
ba6907515e fix: use xz compression when building deb package (#230471) 2024-10-04 13:25:06 +02:00
Robo
8422282f04 chore: update to Electron 32 (#218245)
* chore: update electron@31.0.1

* chore: bump electron@31.1.0

* chore: update sysroot download url

* chore: bump electron@31.3.0

* chore: update rpm deps for arm64

* chore: bump electron@31.4.0

* chore: bump electron@32.0.0

* fix: compile stage

* fix: install on windows

* fix: override node-addon-api to support c++20 with gcc-11

Brings in fed13534c5
to support compilation with gcc>=11

* ci: use gnu++2a for gcc <=9

* ci: fix sdl pipeline

* ci: web stage

* ci: fix alpine stage

* ci: fix legacy server stage

* fix: node-addon-api resolution for remote

* ci: fix legacy arm stage

* chore: sync npm dependencies

* chore: bump electron@32.0.1

* ci: fix monaco editor checks

* ci: fix legacy server stage

* ci: possible concurrent execution

* ci: use foreground scripts on windows

* chore: bump electron@32.0.2

* fix: compilation

Refs 204011c3f2

* Revert "wco - hardcode devtools location on Linux (#227084)"

This reverts commit dfb96d11d3.

* chore: bump electron@32.1.2

* ci: remove workaround for missing c++20 flags

* ci: use gnu++2a for gcc <=9

* chore: bump distro

* ci: use gnu++17 for legacy arm servers

* fixup! override node-addon-api to support c++20 with gcc-11

Scope to only the affected package versions

* chore: bump @vscode/policy-watcher

* chore: bump @vscode/sqlite3

* chore: bump node.js v20.17.0

* chore: bump distro

* chore: update @vscode/windows-ca-certs

* chore: update builds for electron@32.1.2

* chore: bump distro

* chore: bump dependencies

* ci: fix arm legacy servers
2024-10-04 19:19:06 +09:00
Johannes
6e4221f0ec remove swc 2024-10-04 11:23:51 +02:00
Benjamin Pasero
332b89f198 esm - remove AMD build scripts, tests and variables 2024-09-27 08:04:01 +02:00
Robo
b5a6aa14a8 feat: switch to npm as default package manager (#226927)
* feat: move from yarn to npm

* chore: skip yarn.lock files

* fix: playwright download

* chore: fix compile and hygiene

* chore: bump vsce@2.17.0

Refs 8b49e9dfdf

* test: update results for bat and sh colorizer tests

* fix: add missing lock files for windows

* fix: switch to legacy-peer-deps

* chore: update markdown-it@14.1.0

Refs 737c95a129

esbuild step in extensions-ci-pr was previously using markdown-it
from root which had userland punycode and was able to compile successfully.

* ci: increase pr timeout for windows integration tests

* chore: fix product build

* build: ignore extension dev dependency for rcedit

* build: fix working directory inside container

* build: fix dependency generation

* npm: update dependencies

* ci: use global npmrc

* ci: update cache

* ci: setup global npmrc for private npm auth

* build: fix extension bundling

* chore: sync npm dependencies

* ci: debug env variables for container

* ci: fix win32 cli pipeline

* build: fix npmrc config usage for build/ and remote/ dirs

* fix: windows build

* fix: container builds

* fix: markdown-language-features tests and bundling

```
[03:58:22] Error: Command failed: /Users/demohan/.nvm/versions/node/v20.15.1/bin/node /Users/demohan/github/vscode/extensions/markdown-language-features/esbuild-notebook.js --outputRoot /Users/demohan/github/vscode/.build/extensions/markdown-language-features
✘ [ERROR] Could not resolve "punycode"

    extensions/markdown-language-features/node_modules/markdown-it/lib/index.js:14:27:
      14 │ var punycode     = require('punycode');
         ╵                            ~~~~~~~~~~

  The package "punycode" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.
```

Adds userland package based on beed9aee2c

* fix: container builds for distro

* chore: update yarn occurrences

* fixup! chore: bump vsce@2.17.0

Uses the closest version to `main` branch that does not
include d3cc84cdec
while still having the fix 8b49e9dfdf

* chore: sync npm dependencies

* chore: sync npm dependencies

* chore: sync npm dependencies

* chore: throw error when yarn is used for installation

* chore: add review feedback

* chore: switch exec => run where needed

* chore: npm sync dependencies

* fix: markdown-language-features bundling

```
✘ [ERROR] Could not resolve "punycode"

    extensions/markdown-language-features/node_modules/markdown-it/lib/index.js:14:27:
      14 │ var punycode     = require('punycode');
         ╵                            ~~~~~~~~~~

  The package "punycode" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.
```

Adds missing userland package based on markdown-it/markdown-it@beed9ae,
can be removed once we update markdown-it >= 14.1.0

* ci: rename no-yarn-lock-changes.yml

* chore: sync npm dependencies

* ci: restore no-yarn-lock-changes.yml

We can disable it in a separate PR to keep the required
checks happy and also need workflow edit perms.

* chore: sync npm dependencies

* ci: rebuild cache

* ci: fix no-package-lock-changes.yml

* chore: bump distro

* chore: rm yarn.lock files

* chore: rm yarn.lock files without dependencies

* chore: add vscode-selfhost-import-aid to postinstall dirs

* chore: bump distro
2024-09-06 22:18:02 +09:00
Kevin Cui
d1eab2bcf4 ci: ensure retry logic consistency (#226038)
Related PR: #171489

Signed-off-by: Kevin Cui <bh@bugs.cc>
2024-09-04 04:51:19 +00:00
Robo
8e0e3bfb18 ci: add retry logic for apt failures (#227420)
Based on https://github.com/actions/runner-images/blob/ubuntu22/20240825.1/images/ubuntu/scripts/build/configure-apt-mock.sh
2024-09-03 06:55:12 +02:00
Robo
adac839773 ci: use sysroots for stripping binaries in rpmbuild (#227397) 2024-09-03 06:01:02 +02:00
Ladislau Szomoru
310f0a3d85 Engineering - upgrade AzureKeyVault task (#227191) 2024-08-30 11:10:44 +02:00
Benjamin Pasero
6b924c5152 ESM merge to main (#227184)
Co-authored-by: Johannes Rieken <jrieken@microsoft.com>
Co-authored-by: Alexandru Dima <alexdima@microsoft.com>
2024-08-30 10:31:46 +02:00
Benjamin Pasero
c3819e438c esm - pass build variable around properly (#225724) 2024-08-16 09:24:34 +02:00
Robo
d550ede2c3 chore: update glibc version checker for remote (#215599) 2024-06-20 04:30:41 +02:00
Robo
e65febca09 ci: use sysroots for oss linux pipeline (#212895)
* ci: use sysroots for oss linux pipeline

* ci: update cache

* ci: cleanup conditions
2024-05-16 17:18:57 +02:00
João Moreno
e5515ac702 Adopt 1ES PT (#207844)
* extend 1es pipeline template

* oops

* fix template references

* argh

* hmm

* hm

* hm

* use outputs for compilation artifact

* this

* use 1ES.PublishPipelineArtifact@1 instead of publish

* more 1ES.PublishPipelineArtifact@1 adoption

* provide windows pool for sdl sources

* sdl

* fix pools

* fix macos

* disable sbom for intermediate artifacts

* use mariner linux

* try inline tsa options

* fix credscan

* hm

* sudo it

* more suppressions

* be explicit with SBOM build drop paths

* fix indentation

* fix file extensions

* fix cli sbom build drop paths

* fix more build

* fix unzip cli

* careful with _manifest in artifacts

* do not close file

* add logging

* debug

* use snapcraft container

* remove size check

* fix macos cli step

* fix snap permissions

* fix macos

* better logs

* fix snap

* more cred scan suppressions

* even more supressiong

* alpine-arm64: try using x64

* Revert "alpine-arm64: try using x64"

This reverts commit bf2003bf60.

* test docker

* I wonder

* logs

* hm

* fix indentation

* hm

* hm

* fix snap finds

* remove auth

* use hostArchitecture

* snap: limit find

* hm

* sudo

* Add validateToolOutput: None to the build pipeline

* bring back sdl-scan

* try all tools: true

* use release

* Update Azure Pipelines YAML file for Linux product build and test

* hm

* hm

* same for win32

* hm

* hm

* Revert "hm"

This reverts commit 1b9dcae85b.

* use branch

* fix template file

* fix template paths
2024-03-21 14:16:32 +01:00
Robo
5e0394c8d1 fix: don't package detection script in legacy server (#208274)
* fix: don't package detection script in legacy server

* chore: address review feedback

* chore: rename download asset name
2024-03-21 14:11:57 +01:00
Robo
1413fc87a7 feat: create and upload legacy remote server (#204139) 2024-03-18 11:09:55 +01:00
Robo
cf7ddbb51d fix: use legacy server as default with additional warnings (#204377)
* ci: switch to glibc 2.17 remote server

* chore: signal user about unsupported connection

* chore: address review comments

* chore: update nodejs build

* chore: bump distro

* chore: lower the minimum requirements

* fix: glibc version check

* chore: remove explicit connection disposal
2024-02-06 11:25:58 +01:00
Robo
1396f9c827 ci: fix debian packaging step failing to download sysroot (#202973) 2024-01-22 08:12:40 +01:00
Robo
e7b5076dd7 refactor: use sysroots on linux prod pipeline (#192513)
* ci: use sysroots on linux prod pipeline to build native modules

* ci: rebuild after electron update
2023-12-12 11:39:00 +01:00
João Moreno
0b111a09a1 Publish to new CDN (#198154) 2023-11-16 07:19:59 +01:00
João Moreno
417e17038c Use NodeJS mirror for more reliable downloads (#196038)
get node from mirror
2023-10-20 09:53:57 +02:00
Raymond Zhao
880cb517e1 Add more SBOMs (#195736) 2023-10-17 15:55:55 -07:00
Raymond Zhao
18a25cd5d6 Clarify pipeline actions (#195197) 2023-10-11 09:05:00 -07:00