diff --git a/.github/instructions/interactive.instructions.md b/.github/instructions/interactive.instructions.md new file mode 100644 index 00000000000..21ed92f6460 --- /dev/null +++ b/.github/instructions/interactive.instructions.md @@ -0,0 +1,51 @@ +--- +applyTo: '**/interactive/**' +description: Architecture documentation for VS Code interactive window component +--- + +# Interactive Window + +The interactive window component enables extensions to offer REPL like experience to its users. VS Code provides the user interface and extensions provide the execution environment, code completions, execution results rendering and so on. + +The interactive window consists of notebook editor at the top and regular monaco editor at the bottom of the viewport. Extensions can extend the interactive window by leveraging the notebook editor API and text editor/document APIs: + +* Extensions register notebook controllers for the notebook document in the interactive window through `vscode.notebooks.createNotebookController`. The notebook document has a special notebook view type `interactive`, which is contributed by the core instead of extensions. The registered notebook controller is responsible for execution. +* Extensions register auto complete provider for the bottom text editor through `vscode.languages.registerCompletionItemProvider`. The resource scheme for the text editor is `interactive-input` and the language used in the editor is determined by the notebook controller contributed by extensions. + +Users can type in code in the text editor and after users pressing `Shift+Enter`, we will insert a new code cell into the notebook document with the content from the text editor. Then we will request execution for the newly inserted cell. The notebook controller will handle the execution just like it's in a normal notebook editor. + +## Interactive Window Registration + +Registering a new editor type in the workbench consists of two steps: + +* Register an editor input factory which is responsible for resolving resources with given `glob` patterns. Here we register an `InteractiveEditorInput` for all resources with `vscode-interactive` scheme. +* Register an editor pane factory for the given editor input type. Here we register `InteractiveEditor` for our own editor input `InteractiveEditorInput`. + +The workbench editor service is not aware of how models are resolved in `EditorInput`, neither how `EditorPane`s are rendered. It only cares about the common states and events on `EditorInput` or `EditorPane`, i.e., display name, capabilities (editable), content change, dirty state change. It's `EditorInput`/`EditorPane`'s responsibility to provide the right info and updates to the editor service. One major difference between Interactive Editor and other editor panes is Interactive Window is never dirty so users never see a dot on the editor title bar. + +![Editor Registration](./resources/interactive/interactive.editor.drawio.svg) + +## Interactive Window Editor Model Resolution + +The `Interactive.open` command will manually create an `InteractiveEditorInput` specific for the Interactive Window and resolving that Input will go through the following workflow: + +The `INotebookEditorModelResolverService` is used to resolve the notebook model. The `InteractiveEditorInput` wraps a `NotebookEditorInput` for the notebook document and manages a separate text model for the input editor. + +When the notebook model is resolved, the `INotebookEditorModelResolverService` uses the working copy infrastructure to create a `IResolvedNotebookEditorModel`. The content is passed through a `NotebookSerializer` from the `INotebookService` to construct a `NotebookTextModel`. + +![Model Resolution](./resources/interactive/interactive.model.resolution.drawio.svg) + +The `FileSystem` provider that is registered for `vscode-interactive` schema will always return an empty buffer for any read, and will drop all write requests as nothing is stored on disk for Interactive Window resources. The `NotebookSerializer` that is registered for the `interactive` viewtype knows to return an empty notebook data model when it deserializes an empty buffer when the model is being resolved. + +Restoring the interactive window happens through the editor serializer (`InteractiveEditorSerializer`), where the notebook data is stored, and can be used to repopulate the `InteractiveEditorInput` without needing to go through the full editor model resolution flow. + +## UI/EH Editor/Document Syncing + +`EditorInput` is responsible for resolving models for the given resources but in Interactive Window it's much simpler as we are not resolving models ourselves but delegating to Notebook and TextEditor. `InteractiveEditorInput` does the coordination job: + +- It wraps a `NotebookEditorInput` via `_notebookEditorInput` for the notebook document (history cells) +- It manages a separate text model via `ITextModelService` for the input editor at the bottom +- The `IInteractiveDocumentService` coordinates between these two parts +- The `IInteractiveHistoryService` manages command history for the input editor + +![Architecture](./resources/interactive/interactive.eh.drawio.svg) diff --git a/.github/instructions/notebook.instructions.md b/.github/instructions/notebook.instructions.md new file mode 100644 index 00000000000..3d78e744d31 --- /dev/null +++ b/.github/instructions/notebook.instructions.md @@ -0,0 +1,109 @@ +--- +applyTo: '**/notebook/**' +description: Architecture documentation for VS Code notebook and interactive window components +--- + +# Notebook Architecture + +This document describes the internal architecture of VS Code's notebook implementation. + +## Model resolution + +Notebook model resolution is handled by `NotebookService`. It resolves notebook models from the file system or other sources. The notebook model is a tree of cells, where each cell has a type (code or markdown) and a list of outputs. + +## Viewport rendering (virtualization) + +The notebook viewport is virtualized to improve performance. Only visible cells are rendered, and cells outside the viewport are recycled. The viewport rendering is handled by `NotebookCellList` which extends `WorkbenchList`. + +![Viewport Rendering](./resources/notebook/viewport-rendering.drawio.svg) + +The rendering has the following steps: + +1. **Render Viewport** - Layout/render only the cells that are in the visible viewport +2. **Render Template** - Each cell type has a template (code cell, markdown cell) that is instantiated via `CodeCellRenderer` or `MarkupCellRenderer` +3. **Render Element** - The cell content is rendered into the template +4. **Get Dynamic Height** - Cell height is computed dynamically based on content (editor lines, outputs, etc.) +5. **Cell Parts Lifecycle** - Each cell has lifecycle parts that manage focus, selection, and other state + +### Cell resize above viewport + +When a cell above the viewport is resized (e.g., output grows), the viewport needs to be updated to maintain scroll position. This is handled by tracking scroll anchors. + +![Cell Resize Above Viewport](./resources/notebook/cell-resize-above-viewport.drawio.svg) + +## Cell Rendering + +The notebook editor renders cells through a contribution system. Cell parts are organized into two categories via `CellPartsCollection`: + +- **CellContentPart** - Non-floating elements rendered inside a cell synchronously to avoid flickering + - `prepareRenderCell()` - Prepare model (no DOM operations) + - `renderCell()` / `didRenderCell()` - Update DOM for the cell + - `unrenderCell()` - Cleanup when cell leaves viewport + - `updateInternalLayoutNow()` - Update layout per cell layout changes + - `updateState()` - Update per cell state change + - `updateForExecutionState()` - Update per execution state change + +- **CellOverlayPart** - Floating elements rendered on top, may be deferred to next animation frame + +Cell parts are located in `view/cellParts/` and contribute to different aspects: +- **Editor** - The Monaco editor for code cells +- **Outputs** - Rendered outputs from code execution +- **Toolbar** - Cell toolbar with actions +- **Status Bar** - Execution status, language info +- **Decorations** - Fold regions, diagnostics, etc. +- **Context Keys** - Cell-specific context key management +- **Drag and Drop** - Cell reordering via `CellDragAndDropController` + +## Focus Tracking + +Focus in the notebook editor is complex because there are multiple focusable elements: + +1. The notebook list itself (`NotebookCellList`) +2. Individual cell containers +3. Monaco editors within cells +4. Output elements (webviews via `BackLayerWebView`) + +The `NotebookEditorWidget` tracks focus state and provides APIs to manage focus across these components. Context keys like `NOTEBOOK_EDITOR_FOCUSED`, `NOTEBOOK_OUTPUT_FOCUSED`, and `NOTEBOOK_OUTPUT_INPUT_FOCUSED` are used to track focus state. + +## Optimizations + +### Output virtualization + +Large outputs are virtualized similar to cells. Only visible portions of outputs are rendered. + +### Cell DOM recycling + +Cell DOM elements are pooled and recycled to reduce DOM operations. When scrolling, cells that move out of the viewport have their templates returned to the pool. Editor instances are managed via `NotebookCellEditorPool`. + +### Webview reuse + +Output webviews are reused across cells when possible to reduce the overhead of creating new webview contexts. The `BackLayerWebView` manages the webview lifecycle. + +--- + +# Find in Notebook Outputs + +The notebook find feature supports searching in both text models and rendered outputs. The find functionality is implemented via `FindModel` and `CellFindMatchModel` classes. + +## Hybrid Find + +For rendered outputs (HTML, images with alt text, etc.), the find uses a hybrid approach: + +1. **Text model search** - Searches cell source code using standard text search via `FindMatch` +2. **DOM search in webview** - Uses `window.find()` to search rendered output content via `CellWebviewFindMatch` + +![Hybrid Find](./resources/notebook/hybrid-find.drawio.svg) + +The hybrid find works by: + +1. First finding matches in text models (cell inputs) - stored in `contentMatches` +2. Then finding matches in rendered outputs via webview - stored in `webviewMatches` +3. Mixing both match types into a unified result set via `CellFindMatchModel` +4. Navigating between matches reveals the appropriate editor or output + +### Implementation details + +- Uses `window.find()` for DOM searching in webview +- Uses `document.execCommand('hiliteColor')` to highlight matches +- Serializes `document.getSelection()` to get match positions +- Creates ranges for current match highlighting diff --git a/.github/instructions/resources/interactive/interactive.editor.drawio.svg b/.github/instructions/resources/interactive/interactive.editor.drawio.svg new file mode 100644 index 00000000000..c6b94798280 --- /dev/null +++ b/.github/instructions/resources/interactive/interactive.editor.drawio.svg @@ -0,0 +1,331 @@ + + + + + + + + + + + +
+
+
+ Notebook Editor Widget +
+
+
+ + + Notebook Editor Widget + + + + + + + +
+
+
+ Code Editor Widget +
+
+
+ + + Code Editor Widget + + + + + + + +
+
+
+ Interactive Editor +
+
+
+ + + Interactive Editor + + + + + + + + +
+
+
+ NotebookService +
+
+
+ + + NotebookService + + + + + + + + +
+
+
+ TextModelResolverService +
+
+
+ + + TextModelResolverService + + + + + + + + + + + + + + +
+
+
+ NotebookTextModel +
+
+
+ + + NotebookTextModel + + + + + + + +
+
+
+ ITextModel +
+
+
+ + + ITextModel + + + + + + + +
+
+
+ Interactive Editor Input +
+
+
+ + + Interactive Editor In... + + + + + + + +
+
+
+ EditorPane +
+
+
+ + + EditorPane + + + + + + + +
+
+
+ EditorInput +
+
+
+ + + EditorInput + + + + + + + + + +
+
+
+ Editor Resolver Service +
+
+
+ + + Editor Resolver Service + + + + + + + +
+
+
+ EditorPane Registry +
+
+
+ + + EditorPane Registry + + + + + + + + + +
+
+
+ Registry Editor Pane +
+
+
+ + + Registry Editor Pane + + + + + + + + + + + +
+
+
+ + registerEditor + +
+
+
+ + + registerEditor + + + + + + + + + +
+
+
+ input: EditorInput +
+
+
+ + + input: EditorInput + + + + + + + + + +
+
+
+ group: IEditorGroup +
+
+
+ + + group: IEditorGroup + + + + + + + + + +
+
+
+ getControl: IEditorControl +
+
+
+ + + getControl: IEditorControl + + + + + + + + + Text is not SVG - cannot display + + + + diff --git a/.github/instructions/resources/interactive/interactive.eh.drawio.svg b/.github/instructions/resources/interactive/interactive.eh.drawio.svg new file mode 100644 index 00000000000..5f9f40795c2 --- /dev/null +++ b/.github/instructions/resources/interactive/interactive.eh.drawio.svg @@ -0,0 +1,244 @@ + + + + + + + + + +
+
+
+ + Ext Host + +
+
+
+ + + Ext Host + + + + + + + +
+
+
+ + UI + +
+
+
+ + + UI + + + + + + + + + + +
+
+
+ Notebook Editor +
+
+
+ + + Notebook Editor + + + + + + + +
+
+
+ Text Editor +
+
+
+ + + Text Editor + + + + + + + +
+
+
+ Interactive Editor +
+
+
+ + + Interactive Editor + + + + + + + + +
+
+
+ NotebookService +
+
+
+ + + NotebookService + + + + + + + + + + +
+
+
+ TextModelResolverService +
+
+
+ + + TextModelResolverService + + + + + + + + + + +
+
+
+ ExthostNotebook +
+
+
+ + + ExthostNotebook + + + + + + + + + + +
+
+
+ ExthostEditors +
+
+
+ + + ExthostEditors + + + + + + + + + + + +
+
+
+ ExthostInteractive +
+
+
+ + + ExthostInteracti... + + + + + + + + + +
+
+
+ NotebookEditor +
+
+
+ + + NotebookE... + + + + + + + + + +
+
+
+ Input +
+
+
+ + + Input + + + + + + + + + Text is not SVG - cannot display + + + + diff --git a/.github/instructions/resources/interactive/interactive.model.resolution.drawio.svg b/.github/instructions/resources/interactive/interactive.model.resolution.drawio.svg new file mode 100644 index 00000000000..61c91b98125 --- /dev/null +++ b/.github/instructions/resources/interactive/interactive.model.resolution.drawio.svg @@ -0,0 +1,352 @@ + + + + + + + + + + +
+
+
+ Read resource +
+
+
+ + + Read resource + + + + + + + + + +
+
+
+ Deserialize NotebookData +
+
+
+ + + Deserialize NotebookData + + + + + + + + + +
+
+
+ creates +
+
+
+ + + creates + + + + + + + + +
+
+
+ creates +
+
+
+ + + creates + + + + + + + + +
+
+
+ creates +
+
+
+ + + creates + + + + + + + + +
+
+
+ NotebookEditorModelResolverService +
+
+
+ + + NotebookEditorMod... + + + + + + + +
+
+
+ + SimpleNotebookEditorModel + +
+
+
+ + + SimpleNoteboookEd... + + + + + + + +
+
+
+ + NotebookFileWorkingCopyModelFactory + +
+
+
+ + + NotebookFileWorki... + + + + + + + +
+
+
+ + WorkingCopyManager + +
+
+
+ + + WorkingCopyManager + + + + + + + + +
+
+
+ creates +
+
+
+ + + creates + + + + + + + +
+
+
+ NotebookService +
+
+
+ + + NotebookService + + + + + + + +
+
+
+ FileService +
+
+
+ + + FileService + + + + + + + +
+
+
+ NotebookTextModel +
+
+
+ + + NotebookTextModel + + + + + + + + +
+
+
+ register FS provider +
+ for vscode-interactive schema +
+
+
+ + + register FS provider... + + + + + + + + +
+
+
+ register notebook serializer +
+ for interactive viewtype +
+
+
+ + + register notebook serializer... + + + + + + + +
+
+
+ interactive.contribution +
+ (startup) +
+
+
+ + + interactive.contributio... + + + + + + + + + + +
+
+
+ InteractiveEditor +
+
+
+ + + InteractiveEditor + + + + + + + + + + +
+
+
+ + NotebookEditorInput + +
+
+
+ + + NotebookEditorInp... + + + + + + + + + Text is not SVG - cannot display + + + + diff --git a/.github/instructions/resources/notebook/cell-resize-above-viewport.drawio.svg b/.github/instructions/resources/notebook/cell-resize-above-viewport.drawio.svg new file mode 100644 index 00000000000..6a2f80fc98d --- /dev/null +++ b/.github/instructions/resources/notebook/cell-resize-above-viewport.drawio.svg @@ -0,0 +1,654 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + +
+
+
+ Webview top -1000 +
+
+
+ + + Webview top -1000 + + + + + + + + + +
+
+
+ Viewport +
+
+
+ + + Viewport + + + + + + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + + + + + + + + + + + + + +
+
+
+ Grow Height by 50 +
+
+
+ + + Grow Height by 50 + + + + + + + +
+
+
+ scrollTop 1000 +
+
+
+ + + scrollTop 1000 + + + + + + + +
+
+
+ scrollTop 1000 +
+
+
+ + + scrollTop 1000 + + + + + + + + + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + + + + + + + + + + + + + +
+
+
+ scrollTop 1050 +
+
+
+ + + scrollTop 1050 + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + + + + + + + + + + + + + +
+
+
+ scrollTop 1050 +
+
+
+ + + scrollTop 1050 + + + + + + + + + +
+
+
+ Adjust top +
+
+
+ + + Adjust top + + + + + + + + + +
+
+
+ UpdateScrollTop +
+
+
+ + + UpdateScrollTop + + + + + + + + + +
+
+
+ Webview top -1000 +
+
+
+ + + Webview top -1000 + + + + + + + +
+
+
+ Webview top -1000 +
+
+
+ + + Webview top -1000 + + + + + + + +
+
+
+ Webview top -1000 +
+
+
+ + + Webview top -1000 + + + + + + + + + + + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + + + + + + + + + + + + + +
+
+
+ scrollTop 1050 +
+
+
+ + + scrollTop 1050 + + + + + + + +
+
+
+ Webview top -950 +
+
+
+ + + Webview top -950 + + + + + + + + + + + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + + + + + + + + + + + + + +
+
+
+ scrollTop 1050 +
+
+
+ + + scrollTop 1050 + + + + + + + +
+
+
+ Webview top -950 +
+
+
+ + + Webview top -950 + + + + + + + + + +
+
+
+ Adjust top +
+
+
+ + + Adjust top + + + + + + + + + + + +
+
+
+ 1 +
+
+
+ + + 1 + + + + + + + +
+
+
+ 2 +
+
+
+ + + 2 + + + + + + + +
+
+
+ 3 +
+
+
+ + + 3 + + + + + + + +
+
+
+ 4 +
+
+
+ + + 4 + + + + + + + +
+
+
+ 4' +
+
+
+ + + 4' + + + + + + + +
+
+
+ 5 +
+
+
+ + + 5 + + + + + + + + + Viewer does not support full SVG 1.1 + + + + diff --git a/.github/instructions/resources/notebook/hybrid-find.drawio.svg b/.github/instructions/resources/notebook/hybrid-find.drawio.svg new file mode 100644 index 00000000000..a2419b58fce --- /dev/null +++ b/.github/instructions/resources/notebook/hybrid-find.drawio.svg @@ -0,0 +1,327 @@ + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ window.find +
+
+
+ + + window.find + + + + + + + + + +
+
+
+ exec("hiliteColor") +
+ findMatchColor +
+
+
+ + + exec("hiliteColor")... + + + + + + + +
+
+
+ Serialize +
+ document.getSelection() +
+
+
+ + + Serialize... + + + + + + + + + + + + + +
+
+
+ Find in Rendered Outputs (Search in DOM) +
+
+
+ + + Find in Rendered Out... + + + + + + + +
+
+
+ Find +
+
+
+ + + Find + + + + + + + +
+
+
+ Find in Text Model +
+
+
+ + + Find in Text Model + + + + + + + +
+
+
+ Mix matches +
+
+
+ + + Mix matches + + + + + + + +
+
+
+ End of Doc +
+
+
+ + + End of Doc + + + + + + + + + +
+
+
+ Webview +
+
+
+ + + Webview + + + + + + + + + +
+
+
+ Find Next +
+
+
+ + + Find Next + + + + + + + + + +
+
+
+ Is Model Match +
+
+
+ + + Is Model Match + + + + + + + +
+
+
+ Reveal Editor +
+
+
+ + + Reveal Editor + + + + + + + +
+
+
+ Y +
+
+
+ + + Y + + + + + + + + + +
+
+
+ document create range +
+
+
+ + + document create range + + + + + + + + + +
+
+
+ "hiliteColor" +
+ currentFindMatchColor +
+
+
+ + + "hiliteColor"... + + + + + + + + + +
+
+
+ Find cell/output container +
+
+
+ + + Find cell/output con... + + + + + + + + + Viewer does not support full SVG 1.1 + + + + diff --git a/.github/instructions/resources/notebook/viewport-rendering.drawio.svg b/.github/instructions/resources/notebook/viewport-rendering.drawio.svg new file mode 100644 index 00000000000..6a51b66c723 --- /dev/null +++ b/.github/instructions/resources/notebook/viewport-rendering.drawio.svg @@ -0,0 +1,521 @@ + + + + + + + + + + +
+
+
+ Render Viewport +
+
+
+ + + Render Viewport + + + + + + + +
+
+
+ Notebook List View +
+
+
+ + + Notebook List View + + + + + + + + + + + + +
+
+
+ Render Template +
+
+
+ + + Render Template + + + + + + + + + + + + +
+
+
+ Render Element +
+
+
+ + + Render Element + + + + + + + + +
+
+
+ Get Dynamic Height +
+
+
+ + + Get Dynamic Height + + + + + + + + + + + + +
+
+
+ Create Cell Templates/Parts +
+
+
+ + + Create Cell Templates/Parts + + + + + + + +
+
+
+ Toolbar +
+
+
+ + + Toolbar + + + + + + + +
+
+
+ Editor +
+
+
+ + + Editor + + + + + + + +
+
+
+ Statusbar +
+
+
+ + + Statusbar + + + + + + + + + + +
+
+
+ Code Cell +
+
+
+ + + Code Cell + + + + + + + + +
+
+
+ Render Cell Parts +
+
+
+ + + Render Cell Parts + + + + + + + + + + + + + + +
+
+
+ CellPart read DOM +
+
+
+ + + CellPart read DOM + + + + + + + +
+
+
+ Update layout info +
+
+
+ + + Update layout info + + + + + + + + + + +
+
+
+ Toolbar.renderCell +
+
+
+ + + Toolbar.renderCell + + + + + + + +
+
+
+ Toolbar.renderCell +
+
+
+ + + Toolbar.renderCell + + + + + + + +
+
+
+ Toolbar.didRenderCell +
+
+
+ + + Toolbar.didRenderCell + + + + + + + +
+
+
+ Toolbar.renderCell +
+
+
+ + + Toolbar.renderCell + + + + + + + +
+
+
+ Toolbar.renderCell +
+
+
+ + + Toolbar.renderCell + + + + + + + +
+
+
+ + Toolbar.prepareLayout + +
+
+
+ + + Toolbar.prepareLay... + + + + + + + + + + + + + + +
+
+
+ Cell Layout Change +
+
+
+ + + Cell Layout Change + + + + + + + + +
+
+
+ Cell Part updateInternalLayoutNow +
+
+
+ + + Cell Part updateInternalLayoutNow + + + + + + + +
+
+
+ Toolbar.renderCell +
+
+
+ + + Toolbar.renderCell + + + + + + + +
+
+
+ Toolbar.renderCell +
+
+
+ + + Toolbar.renderCell + + + + + + + +
+
+
+ + Toolbar.updateInternalLayoutNow + +
+
+
+ + + Toolbar.updateInternalLayout... + + + + + + + +
+
+
+ Next Frame +
+
+
+ + + Next Frame + + + + + + + +
+
+
+ + DOM Read + +
+
+
+ + + DOM Read + + + + + + + +
+
+
+ + DOM Write + +
+
+
+ + + DOM Write + + + + + + + + + Viewer does not support full SVG 1.1 + + + +