Microsoft/vscode
1
0
Fork 0
mirror of https://github.com/microsoft/vscode.git synced 2025-12-24 12:19:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

JS/TS package acquisition (#184438)

Browse Source 
* Experiment with adding ata using `@types` packages shipped in an extension

* Use own file system instead of `https`

* JS/TS type support on web

* Tsconfig needs esModuleInterop not module:nodenext

We actually want webpack to emit commonjs, but need to write ES default
imports to use node-maintainer

* fix package.json indentation

* Adding setting to disable web type acquisition

* Fix merge of yarn lock

* Fixing merge errors

* Fixing errors

* Pick up package externally

* Fixing conflicts

* Bump version

---------

Co-authored-by: Kat Marchán <kzm@zkat.tech>
Co-authored-by: Nathan Shively-Sanders <293473+sandersn@users.noreply.github.com>
This commit is contained in:
Matt Bierner
2023-08-28 00:49:40 -07:00
committed by GitHub
parent 1a99420d57
commit 45e2e0bfd0
17 changed files with 1057 additions and 271 deletions
Download Patch File Download Diff File Expand all files Collapse all files

177
extensions/typescript-language-features/web/README.md
View File

@@ -1,156 +1,45 @@
# vscode-wasm-typescript
Language server host for typescript using vscode's sync-api in the browser
Language server host for typescript using vscode's sync-api in the browser.
## TODOs
## Getting up and running
### Prototype
To test this out, you'll need three shells:
- [x] get semantic diagnostics rendering squigglies
- typescriptserviceclient.ts has some functions that look at `scheme` to determine some features (hasCapabilityForResource) (also getWorkspaceRootForResource)
- known schemes are in utils/fileSchemes.ts, but don't include vscode-test-web
- adding vscode-test-web in a couple places didn't help, maybe I need to be hackier
- nope, another predicate is `isWeb`, so I had to change place(s) it's used too
- [x] cancellation
1. `yarn watch` for vscode itself
2. `yarn watch-web` for the web side
3. `node <root>/scripts/code-web.js --coi`
### Cleanup
The last command will open a browser window. You'll want to add `?vscode-coi=`
to the end. This is for enabling shared array buffers. So, for example:
`http://localhost:8080/?vscode-coi=`.
- [x] point webpack hack to node_modules; link those files to locally built ones
- [x] create one or more MessageChannels for various communication
- [x] shut down normal listener
- starting the server currently crashes because ts.sys isn't defined -- I think it's a race condition.
In any case it'll need to get shut down before then, which may not be possible without changing Typescript.
- LATER: Turns out you can skip the existing server by depending on tsserverlibrary instead of tsserver.
- [x] figure out a webpack-native way to generate tsserver.web.js if possible
- [x] path rewriting is pretty loosey-goosey; likely to be incorrect some of the time
- invert the logic from TypeScriptServiceClient.normalizedPath for requests
- invert the function from webServer.ts for responses (maybe)
- something with getWorkspaceRootForResource (or anything else that checks `resouce.scheme`)
- [x] put files one level down from virtual root
- [x] fill in missing environment files like lib.dom.d.ts
- toResource's isWeb branch *probably* knows where to find this, just need to put it in the virtual FS
- I guess during setup in serverProcess.browser.ts.
- Not sure whether it needs to have the data or just a fs entry.
- Wait, I don't know how files get added to the FS normally.
- [x] cancellation should only retain one cancellation checker
- the one that matches the current request id
- but that means tracking (or retrieving from tsserver) the request id (aka seq?)
- and correctly setting/resetting it on the cancellation token too.
- I looked at the tsserver code. I think the web case is close to the single-pipe node case,
so I just require that requestId is set in order to call the *current* cancellation checker.
- Any incoming message with a cancellation checker will overwrite the current one.
- [x] Cancellation code in vscode is suspiciously prototypey.
- Specifically, it adds the vscode-wasm cancellation to original cancellation code, but should actually switch to the former for web only.
- looks like `isWeb()` is a way to check for being on the web
- [x] create multiple watchers
- on-demand instead of watching everything and checking on watch firing
- [x] get file watching to work
- it could *already* work, I just don't know how to test it
- look at extensions/markdown-language-features/src/client/fileWatchingManager.ts to see if I can use that
- later: it is OK. its main difference is that you can watch files in not-yet-created directories, and it maintains
a web of directory watches that then check whether the file is eventually created.
- even later: well, it works even though it is similar to my code.
I'm not sure what is different.
- [x] copy fileWatchingManager.ts to web/ ; there's no sharing code between extensions
- [x] Find out scheme the web actually uses instead of vscode-test-web (or switch over entirely to isWeb)
- [x] Need to parse and pass args through so that the syntax server isn't hard-coded to actually be another semantic server
- [x] think about implementing all the other ServerHost methods
- [x] copy importPlugin from previous version of webServer.ts
- [x] also copy details from
- previous implementation (although it's syntax-only so only covers part)
- node implementation in typescript proper
- [x] make realpath support symlinks similarly to node's realpath.
- Johannes says that the filesystem automatically follows symlinks,
so I don't think this is needed.
- [x] organise webServer.ts into multiple files
- OR at least re-arrange it so the diff with the previous version is smaller
- split it into multiple files after the initial PR
- [x] clear out TODOs
- [x] add semicolons everywhere; vscode's lint doesn't seem to complain, but the code clearly uses them
- [x] Further questions about host methods based on existing implementations
- `require` -- is this needed? In TS, it's only used in project system
- `trace` -- is this needed? In TS, it's only used in project system
- `useCaseSensitiveFileNames` -- old version says 'false' is the
safest option, but the virtual fs is case sensitive. Is the old
version still better?
- `writeOutputIsTTY` -- I'm using apiClient.vscode.terminal.write -- is it a tty?
- `getWidthOfTerminal` -- I don't know where to find this on apiClient.vscode.terminal either
- `clearScreen` -- node version writes \x1BC to the terminal. Would
this work for vscode?
- `readFile/writeFile` -- TS handles utf8, utf16le and manually
converts big-endian to utf16 little-endian. How does the in-memory
filesystem handle this? There's no place to specify encoding. (And
`writeFile` currently ignores the flag to write a BOM.)
- `resolvePath` -- node version uses path.resolve. Is it OK to use
that? Or should I re-implement it? Just use identity like the old
web code?
- `getDirectories`/`readDirectory`
- the node code manually skips '.' and '..' in the array returned by
readDirectory. Is this needed?
- `createSHA256Hash` -- the browser version is async, so I skipped it
- `realpath` -- still skips symlinks, I need to figure out what node does
### Working on type acquisition
### Bugs
In order to work with web's new type acquisition, you'll need to enable
`TypeScript > Experimental > Tsserver > Web: Enable Project Wide Intellisense`
in your VS Code options (`Ctrl-,`), you may need to reload the page.
- [x] Response `seq` is always 0.
- [ ] current method of encoding /scheme/authority means that (node) module resolution looks for /scheme/node_modules and /node_modules
- even though they can't possibly exist
- probably not a problem though
- [x] problems pane doesn't clear problems issued on tsconfig.
- This is a known problem in normal usage as well.
- [x] renaming a file throws a No Project error to the console.
- [x] gotodef in another file throws and the editor has a special UI for it.
- definitionProviderBase.getSymbolLocations calls toOpenedFilePath which eventually calls the new / code
- then it calls client.execute which appears to actually request/response to the tsserver
- then the response body is mapped over location.file >> client.toResource >> fromTextSpan
- toResource has isWeb support, as well as (now unused) inMemoryResourcePrefix support
- so I can just redo whatever that did and it'll be fine
This happens when working in a regular `.js` file on a dependency without
declared types. You should be able to open `file.js` and write something like
`import lodash from 'lodash';` at the top of the file and, after a moment, get
types and other intellisense features (like Go To Def/Source Def) working as
expected. This scenario works off Tsserver's own Automatic Type Acquisition
capabilities, and simulates a "global" types cache stored at
`/vscode-global-typings/ts-nul-authority/project`, which is backed by an
in-memory `MemFs` `FileSystemProvider`.
### Done
### Simulated `node_modules`
- [x] need to update 0.2 -> 0.7.* API (once it's working properly)
- [x] including reshuffling the webpack hack if needed
- [x] need to use the settings recommended by Sheetal
- [x] ProjectService always requests a typesMap.json at the cwd
- [x] sync-api-client says fs is rooted at memfs:/sample-folder; the protocol 'memfs:' is confusing our file parsing I think
- [x] nothing ever seems to find tsconfig.json
- [x] messages aren't actually coming through, just the message from the first request
- fixed by simplifying the listener setup for now
- [x] once messages work, you can probably log by postMessage({ type: 'log', body: "some logging text" })
- [x] implement realpath, modifiedtime, resolvepath, then turn semantic mode on
- [x] file watching implemented with saved map of filename to callback, and forwarding
For regular `.ts` files, instead of going through Tsserver's type acquisition,
a separate `AutoInstallerFs` is used to create a "virtual" `node_modules` that
extracts desired packages on demand, to an underlying `MemFs`. This will
happen any time a filesystem operation is done inside a `node_modules` folder
across any project in the workspace, and will use the "real" `package.json`
(and, if present, `package-lock.json`) to resolve the dependency tree.
### Also
- [ ] ATA will eventually need a host interface, or an improvement of the existing one (?)
## Notes
messages received by extension AND host use paths like ^/memfs/ts-nul-authority/sample-folder/file.ts
- problem: pretty sure the extension doesn't know what to do with that: it's not putting down error spans in file.ts
- question: why is the extension requesting quickinfo in that URI format? And it works! (probably because the result is a tooltip, not an in-file span)
- problem: weird concatenations with memfs:/ in the middle
- problem: weird concatenations with ^/memfs/ts-nul-authority in the middle
question: where is the population of sample-folder with a bunch of files happening?
question: Is that location writable while it's running?
but readFile is getting called with things like memfs:/sample-folder/memfs:/typesMap.json
directoryExists with /sample-folder/node_modules/@types and /node_modules/@types
same for watchDirectory
watchDirectory with /sample-folder/^ and directoryExists with /sample-folder/^/memfs/ts-nul-authority/sample-folder/workspaces/
watchFile with /sample-folder/memfs:/sample-folder/memfs:/lib.es2020.full.d.ts
### LATER
OK, so the paths that tsserver has look like this: ^/scheme/mount/whatever.ts
but the paths the filesystem has look like this: scheme:/whatever.ts (not sure about 'mount', that's only when cloning from the fs)
so you have to shave off the scheme that the host combined with the path and put on the scheme that the vfs is using.
### LATER 2
Some commands ask for getExecutingFilePath or getCurrentDirectory and cons up a path themselves.
This works, because URI.from({ scheme, path }) matches what the fs has in it
Problem: In *some* messages (all?), vscode then refers to /x.ts and ^/vscode-test-web/mount/x.ts (or ^/memfs/ts-nul-authority/x.ts)
A fallback is then set up such that when a URI like
`memfs:/path/to/node_modules/lodash/lodash.d.ts` is accessed, that gets
redirected to
`vscode-node-modules:/ts-nul-authority/memfs/ts-nul-authority/path/to/node_modules/lodash/lodash.d.ts`,
which will be sent to the `AutoInstallerFs`.

78
extensions/typescript-language-features/web/jsTyping.ts Normal file
View File

@@ -0,0 +1,78 @@
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
/// Utilities copied from ts.JsTyping internals
export const enum NameValidationResult {
Ok,
EmptyName,
NameTooLong,
NameStartsWithDot,
NameStartsWithUnderscore,
NameContainsNonURISafeCharacters
}
type PackageNameValidationResult = NameValidationResult | ScopedPackageNameValidationResult;
interface ScopedPackageNameValidationResult {
readonly name: string;
readonly isScopeName: boolean;
readonly result: NameValidationResult;
}
enum CharacterCodes {
_ = 0x5F,
dot = 0x2E,
}
const maxPackageNameLength = 214;
// Validates package name using rules defined at https://docs.npmjs.com/files/package.json
// Copied from typescript/jsTypings.ts
export function validatePackageNameWorker(packageName: string, supportScopedPackage: true): ScopedPackageNameValidationResult;
export function validatePackageNameWorker(packageName: string, supportScopedPackage: false): NameValidationResult;
export function validatePackageNameWorker(packageName: string, supportScopedPackage: boolean): PackageNameValidationResult {
if (!packageName) {
return NameValidationResult.EmptyName;
}
if (packageName.length > maxPackageNameLength) {
return NameValidationResult.NameTooLong;
}
if (packageName.charCodeAt(0) === CharacterCodes.dot) {
return NameValidationResult.NameStartsWithDot;
}
if (packageName.charCodeAt(0) === CharacterCodes._) {
return NameValidationResult.NameStartsWithUnderscore;
}
// check if name is scope package like: starts with @ and has one '/' in the middle
// scoped packages are not currently supported
if (supportScopedPackage) {
const matches = /^@([^/]+)\/([^/]+)$/.exec(packageName);
if (matches) {
const scopeResult = validatePackageNameWorker(matches[1], /*supportScopedPackage*/ false);
if (scopeResult !== NameValidationResult.Ok) {
return { name: matches[1], isScopeName: true, result: scopeResult };
}
const packageResult = validatePackageNameWorker(matches[2], /*supportScopedPackage*/ false);
if (packageResult !== NameValidationResult.Ok) {
return { name: matches[2], isScopeName: false, result: packageResult };
}
return NameValidationResult.Ok;
}
}
if (encodeURIComponent(packageName) !== packageName) {
return NameValidationResult.NameContainsNonURISafeCharacters;
}
return NameValidationResult.Ok;
}
export interface TypingResolutionHost {
directoryExists(path: string): boolean;
fileExists(fileName: string): boolean;
readFile(path: string, encoding?: string): string | undefined;
readDirectory(rootDir: string, extensions: readonly string[], excludes: readonly string[] | undefined, includes: readonly string[] | undefined, depth?: number): string[];
}

3
extensions/typescript-language-features/web/tsconfig.json
View File

@@ -2,8 +2,7 @@
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "../../out",
"module": "nodenext",
"moduleDetection": "legacy",
"esModuleInterop": true,
"experimentalDecorators": true,
"types": [
"node"

213
extensions/typescript-language-features/web/typingsInstaller.ts Normal file
View File

@@ -0,0 +1,213 @@
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
/*
* This file implements the global typings installer API for web clients. It
* uses [nassun](https://docs.rs/nassun) and
* [node-maintainer](https://docs.rs/node-maintainer) to install typings
* in-memory (and maybe eventually cache them in IndexedDB?).
*
* Implementing a typings installer involves implementing two parts:
*
* -> ITypingsInstaller: the "top level" interface that tsserver uses to
* request typings. Implementers of this interface are what actually get
* passed to tsserver.
*
* -> TypingsInstaller: an abstract class that implements a good chunk of
* the "generic" functionality for what ITypingsInstaller needs to do. For
* implementation detail reasons, it does this in a "server/client" model of
* sorts. In our case, we don't need a separate process, or even _quite_ a
* pure "server/client" model, so we play along a bit for the sake of reusing
* the stuff the abstract class is already doing for us.
*/
import { PackageManager, PackageType } from '@vscode/ts-package-manager';
import { join } from 'path';
import * as ts from 'typescript/lib/tsserverlibrary';
import { NameValidationResult, validatePackageNameWorker } from './jsTyping';
type InstallerResponse = ts.server.PackageInstalledResponse | ts.server.SetTypings | ts.server.InvalidateCachedTypings | ts.server.BeginInstallTypes | ts.server.EndInstallTypes | ts.server.WatchTypingLocations;
/**
* The "server" part of the "server/client" model. This is the part that
* actually gets instantiated and passed to tsserver.
*/
export default class WebTypingsInstallerClient implements ts.server.ITypingsInstaller {
private projectService: ts.server.ProjectService | undefined;
private requestedRegistry = false;
private typesRegistryCache: Map<string, ts.MapLike<string>> = new Map();
private readonly server: Promise<WebTypingsInstallerServer>;
constructor(
private readonly fs: ts.server.ServerHost,
readonly globalTypingsCacheLocation: string,
) {
this.server = WebTypingsInstallerServer.initialize(
(response: InstallerResponse) => this.handleResponse(response),
this.fs,
globalTypingsCacheLocation
);
}
/**
* TypingsInstaller expects a "server/client" model, and as such, some of
* its methods are implemented in terms of sending responses back to a
* client. This method is a catch-all for those responses generated by
* TypingsInstaller internals.
*/
private async handleResponse(response: InstallerResponse): Promise<void> {
switch (response.kind) {
case 'action::packageInstalled':
case 'action::invalidate':
case 'action::set':
this.projectService!.updateTypingsForProject(response);
break;
case 'event::beginInstallTypes':
case 'event::endInstallTypes':
// Don't care.
break;
default:
throw new Error(`unexpected response: ${response}`);
}
}
// NB(kmarchan): this is a code action that expects an actual NPM-specific
// installation. We shouldn't mess with this ourselves.
async installPackage(_options: ts.server.InstallPackageOptionsWithProject): Promise<ts.ApplyCodeActionCommandResult> {
throw new Error('not implemented');
}
// NB(kmarchan): As far as I can tell, this is only ever used for
// completions?
isKnownTypesPackageName(packageName: string): boolean {
console.log('isKnownTypesPackageName', packageName);
const looksLikeValidName = validatePackageNameWorker(packageName, true);
if (looksLikeValidName.result !== NameValidationResult.Ok) {
return false;
}
if (this.requestedRegistry) {
return !!this.typesRegistryCache && this.typesRegistryCache.has(packageName);
}
this.requestedRegistry = true;
this.server.then(s => this.typesRegistryCache = s.typesRegistry);
return false;
}
enqueueInstallTypingsRequest(p: ts.server.Project, typeAcquisition: ts.TypeAcquisition, unresolvedImports: ts.SortedReadonlyArray<string>): void {
console.log('enqueueInstallTypingsRequest', typeAcquisition, unresolvedImports);
const req = ts.server.createInstallTypingsRequest(p, typeAcquisition, unresolvedImports);
this.server.then(s => s.install(req));
}
attach(projectService: ts.server.ProjectService): void {
this.projectService = projectService;
}
onProjectClosed(_projectService: ts.server.Project): void {
// noop
}
}
/**
* Internal implementation of the "server" part of the "server/client" model.
* This takes advantage of the existing TypingsInstaller to reuse a lot of
* already-implemented logic around package installation, but with
* installation details handled by Nassun/Node Maintainer.
*/
class WebTypingsInstallerServer extends ts.server.typingsInstaller.TypingsInstaller {
private static readonly typesRegistryPackageName = 'types-registry';
private constructor(
override typesRegistry: Map<string, ts.MapLike<string>>,
private readonly handleResponse: (response: InstallerResponse) => void,
fs: ts.server.ServerHost,
private readonly packageManager: PackageManager,
globalTypingsCachePath: string,
) {
super(fs, globalTypingsCachePath, join(globalTypingsCachePath, 'fakeSafeList') as ts.Path, join(globalTypingsCachePath, 'fakeTypesMapLocation') as ts.Path, Infinity);
}
/**
* Because loading the typesRegistry is an async operation for us, we need
* to have a separate "constructor" that will be used by
* WebTypingsInstallerClient.
*
* @returns a promise that resolves to a WebTypingsInstallerServer
*/
static async initialize(
handleResponse: (response: InstallerResponse) => void,
fs: ts.server.ServerHost,
globalTypingsCachePath: string,
): Promise<WebTypingsInstallerServer> {
const pm = new PackageManager(fs);
const pkgJson = join(globalTypingsCachePath, 'package.json');
if (!fs.fileExists(pkgJson)) {
fs.writeFile(pkgJson, '{"private":true}');
}
const resolved = await pm.resolveProject(globalTypingsCachePath, {
addPackages: [this.typesRegistryPackageName]
});
await resolved.restore();
const registry = new Map<string, ts.MapLike<string>>();
const indexPath = join(globalTypingsCachePath, 'node_modules/types-registry/index.json');
const index = WebTypingsInstallerServer.readJson(fs, indexPath);
for (const [packageName, entry] of Object.entries(index.entries)) {
registry.set(packageName, entry as ts.MapLike<string>);
}
console.log('ATA registry loaded');
return new WebTypingsInstallerServer(registry, handleResponse, fs, pm, globalTypingsCachePath);
}
/**
* Implements the actual logic of installing a set of given packages. It
* does this by looking up the latest versions of those packages using
* Nassun, then handing Node Maintainer the updated package.json to run a
* full install (modulo existing lockfiles, which can make this faster).
*/
protected override installWorker(requestId: number, packageNames: string[], cwd: string, onRequestCompleted: ts.server.typingsInstaller.RequestCompletedAction): void {
console.log('installWorker', requestId, cwd);
(async () => {
try {
const resolved = await this.packageManager.resolveProject(cwd, {
addPackages: packageNames,
packageType: PackageType.DevDependency
});
await resolved.restore();
onRequestCompleted(true);
} catch (e) {
onRequestCompleted(false);
}
})();
}
/**
* This is a thing that TypingsInstaller uses internally to send
* responses, and we'll need to handle this in the Client later.
*/
protected override sendResponse(response: InstallerResponse): void {
this.handleResponse(response);
}
/**
* What it says on the tin. Reads a JSON file from the given path. Throws
* if the file doesn't exist (as opposed to returning `undefined`, like
* fs.readFile does).
*/
private static readJson(fs: ts.server.ServerHost, path: string): any {
const data = fs.readFile(path);
if (!data) {
throw new Error('Failed to read file: ' + path);
}
return JSON.parse(data.trim());
}
}

263
extensions/typescript-language-features/web/webServer.ts
View File

@@ -5,10 +5,12 @@
/// <reference lib='webworker.importscripts' />
/// <reference lib='webworker' />
import * as ts from 'typescript/lib/tsserverlibrary';
import { ApiClient, FileType, Requests } from '@vscode/sync-api-client';
import { ApiClient, FileStat, FileSystem, FileType, Requests } from '@vscode/sync-api-client';
import { ClientConnection } from '@vscode/sync-api-common/browser';
import { basename } from 'path';
import * as ts from 'typescript/lib/tsserverlibrary';
import { URI } from 'vscode-uri';
import WebTypingsInstaller from './typingsInstaller';
// GLOBALS
const watchFiles: Map<string, { path: string; callback: ts.FileWatcherCallback; pollingInterval?: number; options?: ts.WatchOptions }> = new Map();
@@ -87,7 +89,7 @@ class AccessOutsideOfRootError extends Error {
type ServerHostWithImport = ts.server.ServerHost & { importPlugin(root: string, moduleName: string): Promise<ts.server.ModuleImportResult> };
function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient: ApiClient | undefined, args: string[], fsWatcher: MessagePort): ServerHostWithImport {
function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient: ApiClient | undefined, args: string[], fsWatcher: MessagePort, enabledExperimentalTypeAcquisition: boolean): ServerHostWithImport {
const currentDirectory = '/';
const fs = apiClient?.vscode.workspace.fileSystem;
let watchId = 0;
@@ -99,7 +101,6 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
const directorySeparator: string = (ts as any).directorySeparator;
const executingFilePath = findArgument(args, '--executingFilePath') || location + '';
const getExecutingDirectoryPath = memoize(() => memoize(() => ensureTrailingDirectorySeparator(getDirectoryPath(executingFilePath))));
// Later we could map ^memfs:/ to do something special if we want to enable more functionality like module resolution or something like that
const getWebPath = (path: string) => path.startsWith(directorySeparator) ? path.replace(directorySeparator, getExecutingDirectoryPath()) : undefined;
const textDecoder = new TextDecoder();
@@ -121,6 +122,8 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
return noopWatcher;
}
console.log('watching file:', path);
logVerbose('fs.watchFile', { path });
let uri: URI;
@@ -132,14 +135,20 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
}
watchFiles.set(path, { path, callback, pollingInterval, options });
watchId++;
fsWatcher.postMessage({ type: 'watchFile', uri, id: watchId });
const watchIds = [++watchId];
fsWatcher.postMessage({ type: 'watchFile', uri: uri, id: watchIds[0] });
if (enabledExperimentalTypeAcquisition && looksLikeNodeModules(path)) {
watchIds.push(++watchId);
fsWatcher.postMessage({ type: 'watchFile', uri: mapUri(uri, 'vscode-node-modules'), id: watchIds[1] });
}
return {
close() {
logVerbose('fs.watchFile.close', { path });
watchFiles.delete(path);
fsWatcher.postMessage({ type: 'dispose', id: watchId });
for (const id of watchIds) {
fsWatcher.postMessage({ type: 'dispose', id });
}
}
};
},
@@ -155,14 +164,16 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
}
watchDirectories.set(path, { path, callback, recursive, options });
watchId++;
const watchIds = [++watchId];
fsWatcher.postMessage({ type: 'watchDirectory', recursive, uri, id: watchId });
return {
close() {
logVerbose('fs.watchDirectory.close', { path });
watchDirectories.delete(path);
fsWatcher.postMessage({ type: 'dispose', id: watchId });
for (const id of watchIds) {
fsWatcher.postMessage({ type: 'dispose', id });
}
}
};
},
@@ -226,14 +237,28 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
}
}
let uri;
try {
// We need to slice the bytes since we can't pass a shared array to text decoder
const contents = fs.readFile(toResource(path)).slice();
return textDecoder.decode(contents);
} catch (error) {
logNormal('Error fs.readFile', { path, error: error + '' });
uri = toResource(path);
} catch (e) {
return undefined;
}
let contents: Uint8Array | undefined;
try {
// We need to slice the bytes since we can't pass a shared array to text decoder
contents = fs.readFile(uri);
} catch (error) {
if (!enabledExperimentalTypeAcquisition) {
return undefined;
}
try {
contents = fs.readFile(mapUri(uri, 'vscode-node-modules'));
} catch (e) {
return undefined;
}
}
return textDecoder.decode(contents.slice());
},
getFileSize(path) {
logVerbose('fs.getFileSize', { path });
@@ -242,12 +267,19 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
throw new Error('not supported');
}
const uri = toResource(path);
let ret = 0;
try {
return fs.stat(toResource(path)).size;
} catch (error) {
logNormal('Error fs.getFileSize', { path, error: error + '' });
return 0;
ret = fs.stat(uri).size;
} catch (_error) {
if (enabledExperimentalTypeAcquisition) {
try {
ret = fs.stat(mapUri(uri, 'vscode-node-modules')).size;
} catch (_error) {
}
}
}
return ret;
},
writeFile(path, data, writeByteOrderMark) {
logVerbose('fs.writeFile', { path });
@@ -260,10 +292,21 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
data = byteOrderMarkIndicator + data;
}
let uri;
try {
fs.writeFile(toResource(path), textEncoder.encode(data));
uri = toResource(path);
} catch (e) {
return;
}
const encoded = textEncoder.encode(data);
try {
fs.writeFile(uri, encoded);
const name = basename(uri.path);
if (uri.scheme !== 'vscode-global-typings' && (name === 'package.json' || name === 'package-lock.json' || name === 'package-lock.kdl')) {
fs.writeFile(mapUri(uri, 'vscode-node-modules'), encoded);
}
} catch (error) {
logNormal('Error fs.writeFile', { path, error: error + '' });
console.error('fs.writeFile', { path, error });
}
},
resolvePath(path: string): string {
@@ -284,12 +327,24 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
return request.status === 200;
}
let uri;
try {
return fs.stat(toResource(path)).type === FileType.File;
} catch (error) {
logNormal('Error fs.fileExists', { path, error: error + '' });
uri = toResource(path);
} catch (e) {
return false;
}
let ret = false;
try {
ret = fs.stat(uri).type === FileType.File;
} catch (_error) {
if (enabledExperimentalTypeAcquisition) {
try {
ret = fs.stat(mapUri(uri, 'vscode-node-modules')).type === FileType.File;
} catch (_error) {
}
}
}
return ret;
},
directoryExists(path: string): boolean {
logVerbose('fs.directoryExists', { path });
@@ -298,10 +353,32 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
return false;
}
let uri;
try {
return fs.stat(toResource(path)).type === FileType.Directory;
} catch (error) {
logNormal('Error fs.directoryExists', { path, error: error + '' });
uri = toResource(path);
} catch (_error) {
return false;
}
let stat: FileStat | undefined = undefined;
try {
stat = fs.stat(uri);
} catch (_error) {
if (enabledExperimentalTypeAcquisition) {
try {
stat = fs.stat(mapUri(uri, 'vscode-node-modules'));
} catch (_error) {
}
}
}
if (stat) {
if (path.startsWith('/https') && !path.endsWith('.d.ts')) {
// TODO: Hack, https "file system" can't actually tell what is a file vs directory
return stat.type === FileType.File || stat.type === FileType.Directory;
}
return stat.type === FileType.Directory;
} else {
return false;
}
},
@@ -341,12 +418,19 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
throw new Error('not supported');
}
const uri = toResource(path);
let s: FileStat | undefined = undefined;
try {
return new Date(fs.stat(toResource(path)).mtime);
} catch (error) {
logNormal('Error fs.getModifiedTime', { path, error: error + '' });
return undefined;
s = fs.stat(uri);
} catch (_e) {
if (enabledExperimentalTypeAcquisition) {
try {
s = fs.stat(mapUri(uri, 'vscode-node-modules'));
} catch (_e) {
}
}
}
return s && new Date(s.mtime);
},
deleteFile(path: string): void {
logVerbose('fs.deleteFile', { path });
@@ -373,13 +457,19 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
base64encode: input => Buffer.from(input).toString('base64'),
};
/** For module resolution only; symlinks aren't supported yet. */
// For module resolution only. `node_modules` is also automatically mapped
// as if all node_modules-like paths are symlinked.
function realpath(path: string): string {
// skip paths without .. or ./ or /.
if (!path.match(/\.\.|\/\.|\.\//)) {
const isNm = looksLikeNodeModules(path) && !path.startsWith('/vscode-global-typings/');
// skip paths without .. or ./ or /. And things that look like node_modules
if (!isNm && !path.match(/\.\.|\/\.|\.\//)) {
return path;
}
const uri = toResource(path);
let uri = toResource(path);
if (isNm) {
uri = mapUri(uri, 'vscode-node-modules');
}
const out = [uri.scheme];
if (uri.authority) { out.push(uri.authority); }
for (const part of uri.path.split('/')) {
@@ -403,31 +493,35 @@ function createServerHost(extensionUri: URI, logger: ts.server.Logger, apiClient
throw new Error('not supported');
}
const uri = toResource(path || '.');
let entries: [string, FileType][] = [];
const files: string[] = [];
const directories: string[] = [];
try {
const uri = toResource(path || '.');
const entries = fs.readDirectory(uri);
const files: string[] = [];
const directories: string[] = [];
for (const [entry, type] of entries) {
// This is necessary because on some file system node fails to exclude
// '.' and '..'. See https://github.com/nodejs/node/issues/4002
if (entry === '.' || entry === '..') {
continue;
}
if (type === FileType.File) {
files.push(entry);
}
else if (type === FileType.Directory) {
directories.push(entry);
}
entries = fs.readDirectory(uri);
} catch (_e) {
try {
entries = fs.readDirectory(mapUri(uri, 'vscode-node-modules'));
} catch (_e) {
}
files.sort();
directories.sort();
return { files, directories };
} catch (e) {
return { files: [], directories: [] };
}
for (const [entry, type] of entries) {
// This is necessary because on some file system node fails to exclude
// '.' and '..'. See https://github.com/nodejs/node/issues/4002
if (entry === '.' || entry === '..') {
continue;
}
if (type === FileType.File) {
files.push(entry);
}
else if (type === FileType.Directory) {
directories.push(entry);
}
}
files.sort();
directories.sort();
return { files, directories };
}
/**
@@ -475,6 +569,10 @@ function looksLikeLibDtsPath(filepath: string) {
return filepath.startsWith('/lib.') && filepath.endsWith('.d.ts');
}
function looksLikeNodeModules(filepath: string) {
return filepath.includes('/node_modules');
}
function filePathToResourceUri(filepath: string): URI | undefined {
const parts = filepath.match(/^\/([^\/]+)\/([^\/]*)(?:\/(.+))?$/);
if (!parts) {
@@ -509,14 +607,15 @@ class WasmCancellationToken implements ts.server.ServerCancellationToken {
}
interface StartSessionOptions {
globalPlugins: ts.server.SessionOptions['globalPlugins'];
pluginProbeLocations: ts.server.SessionOptions['pluginProbeLocations'];
allowLocalPluginLoads: ts.server.SessionOptions['allowLocalPluginLoads'];
useSingleInferredProject: ts.server.SessionOptions['useSingleInferredProject'];
useInferredProjectPerProjectRoot: ts.server.SessionOptions['useInferredProjectPerProjectRoot'];
suppressDiagnosticEvents: ts.server.SessionOptions['suppressDiagnosticEvents'];
noGetErrOnBackgroundUpdate: ts.server.SessionOptions['noGetErrOnBackgroundUpdate'];
serverMode: ts.server.SessionOptions['serverMode'];
readonly globalPlugins: ts.server.SessionOptions['globalPlugins'];
readonly pluginProbeLocations: ts.server.SessionOptions['pluginProbeLocations'];
readonly allowLocalPluginLoads: ts.server.SessionOptions['allowLocalPluginLoads'];
readonly useSingleInferredProject: ts.server.SessionOptions['useSingleInferredProject'];
readonly useInferredProjectPerProjectRoot: ts.server.SessionOptions['useInferredProjectPerProjectRoot'];
readonly suppressDiagnosticEvents: ts.server.SessionOptions['suppressDiagnosticEvents'];
readonly noGetErrOnBackgroundUpdate: ts.server.SessionOptions['noGetErrOnBackgroundUpdate'];
readonly serverMode: ts.server.SessionOptions['serverMode'];
readonly disableAutomaticTypingAcquisition: boolean;
}
class WorkerSession extends ts.server.Session<{}> {
@@ -526,17 +625,20 @@ class WorkerSession extends ts.server.Session<{}> {
constructor(
host: ts.server.ServerHost,
fs: FileSystem | undefined,
options: StartSessionOptions,
public readonly port: MessagePort,
private readonly port: MessagePort,
logger: ts.server.Logger,
hrtime: ts.server.SessionOptions['hrtime']
) {
const cancellationToken = new WasmCancellationToken();
const typingsInstaller = options.disableAutomaticTypingAcquisition || !fs ? ts.server.nullTypingsInstaller : new WebTypingsInstaller(host, '/vscode-global-typings/ts-nul-authority/projects');
super({
host,
cancellationToken,
...options,
typingsInstaller: ts.server.nullTypingsInstaller, // TODO: Someday!
typingsInstaller,
byteLength: () => { throw new Error('Not implemented'); }, // Formats the message text in send of Session which is overridden in this class so not needed
hrtime,
logger,
@@ -671,7 +773,7 @@ async function initializeSession(args: string[], extensionUri: URI, ports: { tss
logger.info(`Version: 0.0.0`);
logger.info(`Arguments: ${args.join(' ')}`);
logger.info(`ServerMode: ${serverMode} unknownServerMode: ${unknownServerMode}`);
const options = {
const options: StartSessionOptions = {
globalPlugins: findArgumentStringArray(args, '--globalPlugins'),
pluginProbeLocations: findArgumentStringArray(args, '--pluginProbeLocations'),
allowLocalPluginLoads: hasArgument(args, '--allowLocalPluginLoads'),
@@ -679,22 +781,27 @@ async function initializeSession(args: string[], extensionUri: URI, ports: { tss
useInferredProjectPerProjectRoot: hasArgument(args, '--useInferredProjectPerProjectRoot'),
suppressDiagnosticEvents: hasArgument(args, '--suppressDiagnosticEvents'),
noGetErrOnBackgroundUpdate: hasArgument(args, '--noGetErrOnBackgroundUpdate'),
serverMode
serverMode,
disableAutomaticTypingAcquisition: hasArgument(args, '--disableAutomaticTypingAcquisition'),
};
let sys: ServerHostWithImport;
let fs: FileSystem | undefined;
if (hasArgument(args, '--enableProjectWideIntelliSenseOnWeb')) {
const enabledExperimentalTypeAcquisition = hasArgument(args, '--experimentalTypeAcquisition');
const connection = new ClientConnection<Requests>(ports.sync);
await connection.serviceReady();
sys = createServerHost(extensionUri, logger, new ApiClient(connection), args, ports.watcher);
const apiClient = new ApiClient(connection);
fs = apiClient.vscode.workspace.fileSystem;
sys = createServerHost(extensionUri, logger, apiClient, args, ports.watcher, enabledExperimentalTypeAcquisition);
} else {
sys = createServerHost(extensionUri, logger, undefined, args, ports.watcher);
sys = createServerHost(extensionUri, logger, undefined, args, ports.watcher, false);
}
setSys(sys);
session = new WorkerSession(sys, options, ports.tsserver, logger, hrtime);
session = new WorkerSession(sys, fs, options, ports.tsserver, logger, hrtime);
session.listen();
}
@@ -743,3 +850,15 @@ const listener = async (e: any) => {
console.error(`unexpected message on main channel: ${JSON.stringify(e)}`);
};
addEventListener('message', listener);
function mapUri(uri: URI, mappedScheme: string): URI {
if (uri.scheme === 'vscode-global-typings') {
throw new Error('can\'t map vscode-global-typings');
}
if (!uri.authority) {
uri = uri.with({ authority: 'ts-nul-authority' });
}
uri = uri.with({ scheme: mappedScheme, path: `/${uri.scheme}/${uri.authority || 'ts-nul-authority'}${uri.path}` });
return uri;
}