Optimize node_modules caching for CI & PR checks (#322074)

* CI: speed up node_modules cache with zstd + shared scripts

Switch the Linux/macOS node_modules cache from single-threaded gzip
(tar -czf) to multi-threaded zstd. The "Create node_modules archive"
step was spending ~5min of single-core gzip on a multi-GB tree on every
cache miss; zstd -T0 uses all cores and decompresses much faster, so
cache-hit jobs benefit too. Windows stays on 7-Zip (already threaded).

Extract the archive/extract commands into shared per-platform scripts
under .github/workflows/node_modules_cache/ (cache.sh / cache.ps1, each
dispatching on an archive|extract argument) so the format and flags live
in one place instead of being duplicated across ~8 workflows. Bump
build/.cachesalt to invalidate existing gzip caches.

Also remove the obsolete extensions/copilot CI workflows
(copilot-setup-steps.yml, ensure-node-modules-cache.yml, pr.yml) and the
unused build/listBuildCacheFiles.js, and drop their now-stale entries
(plus lit-html and signals-core) from .eslint-allowed-javascript-files.

* ci: seed copilot node_modules cache on main and rename cache keys

Add copilot-linux and copilot-windows jobs to pr-node-modules.yml so the
copilot node_modules cache is populated on main. Rename the copilot cache
keys to copilot-node_modules-linux / copilot-node_modules-windows in pr.yml.

* ci: extract node_modules cache into composite actions

Factor the repeated node_modules cache plumbing into two local composite
actions, restore-node-modules and save-node-modules, and migrate all
workflows that used the cache.sh/cache.ps1 archive flow (pr, pr-node-modules,
pr-{linux,darwin,win32}-test, copilot-setup-steps, component-fixtures,
css-order-scan).

- restore-node-modules computes the key, restores the cache, optionally
  extracts on a hit, and exports the resolved key via $GITHUB_ENV.
- save-node-modules archives node_modules and saves it to the cache, reusing
  the key exported by restore so callers don't repeat the prefix.
- Bespoke install steps stay in the workflows, so per-job env/secrets never
  cross the action boundary.
- Only seed the cache on branch pushes (component-fixtures skips PRs, whose
  caches aren't shared).

* save the node_modules cache for now to test it

* ci: fix node_modules cache save dropping the archive

cache.sh wrote its archive as cache.tzst, but actions/cache reserves that
name for its own tarball and passes --exclude cache.tzst, so our archive was
excluded and an empty (~200 B) cache was saved on Linux/macOS. Rename the
archive to node-modules.tzst and bump build/.cachesalt to invalidate the
broken cache entries.

* empty commit

* Remove again saving to the node modules cache from PR steps
This commit is contained in:
Alexandru Dima
2026-06-19 17:21:43 +02:00
committed by GitHub
parent f736b7d66d
commit 37e7e85b10
21 changed files with 313 additions and 689 deletions
@@ -0,0 +1,35 @@
# node_modules cache scripts
Shared helpers used by the GitHub Actions workflows to store and restore the
`node_modules` cache. They exist so the archive format and compression flags
live in a single place instead of being duplicated across every workflow.
Each script takes an action argument:
| Script | Platform | `archive` | `extract` |
| ------ | -------- | --------- | --------- |
| `cache.sh <action>` | Linux / macOS | Create `node-modules.tzst` (cache miss) | Restore `node-modules.tzst` (cache hit) |
| `cache.ps1 <action>` | Windows | Create `cache.7z` (cache miss) | Restore `cache.7z` (cache hit) |
Linux/macOS use multi-threaded `zstd` (`node-modules.tzst`); Windows uses 7-Zip
(`cache.7z`). The archive must NOT be named `cache.tzst`: `actions/cache` names
its own tarball `cache.tzst` and passes `--exclude cache.tzst`, which would drop
our archive and save an empty cache. The two families are not interchangeable,
so the cache key already encodes the OS (`node_modules-linux-*`,
`node_modules-windows-*`, …).
Example usage from a workflow step:
```yaml
- name: Extract node_modules cache
if: steps.cache-node-modules.outputs.cache-hit == 'true'
run: ./.github/workflows/node_modules_cache/cache.sh extract
- name: Create node_modules archive
if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: ./.github/workflows/node_modules_cache/cache.sh archive
```
If you change the archive format or flags, update both the `archive` and
`extract` branch of the script, and bump `build/.cachesalt` to invalidate
existing caches.
@@ -0,0 +1,27 @@
# Store or restore the node_modules cache on Windows.
#
# Usage:
# cache.ps1 archive # create .build/node_modules_cache/cache.7z (cache miss)
# cache.ps1 extract # restore .build/node_modules_cache/cache.7z (cache hit)
#
# Uses 7-Zip. The archive and extract format must stay compatible; if you change
# the format here, bump build/.cachesalt to invalidate old caches.
param(
[Parameter(Mandatory = $true)]
[ValidateSet("archive", "extract")]
[string]$Action
)
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
switch ($Action) {
"archive" {
exec { node build/azure-pipelines/common/listNodeModules.ts .build/node_modules_list.txt }
exec { mkdir -Force .build/node_modules_cache }
exec { 7z.exe a .build/node_modules_cache/cache.7z -mx3 `@.build/node_modules_list.txt }
}
"extract" {
exec { 7z.exe x .build/node_modules_cache/cache.7z -aoa }
}
}
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# Store or restore the node_modules cache on Linux / macOS.
#
# Usage:
# cache.sh archive # create .build/node_modules_cache/node-modules.tzst (cache miss)
# cache.sh extract # restore .build/node_modules_cache/node-modules.tzst (cache hit)
#
# Uses multi-threaded zstd. The archive must NOT be named cache.tzst because
# actions/cache reserves that name and excludes it from the saved cache. The
# archive and extract flags must stay compatible; if you change the format
# here, bump build/.cachesalt to invalidate old caches.
set -e
case "$1" in
archive)
node build/azure-pipelines/common/listNodeModules.ts .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar --use-compress-program='zstd -T0 -3' -cf .build/node_modules_cache/node-modules.tzst --files-from .build/node_modules_list.txt
;;
extract)
tar --use-compress-program='zstd -d -T0' -xf .build/node_modules_cache/node-modules.tzst
;;
*)
echo "Usage: $0 {archive|extract}" >&2
exit 1
;;
esac