lm: a second rendition of returning data from LM tools (#225634)

* lm: a second rendition of returning data from LM tools

This is an alternative to #225454. It allows the tool caller to pass
through token budget and counting information to the tool, and the
tool can then 'do its thing.'

Most of the actual implementation is in prompt-tsx with a new method to
render elements into a JSON-serializable form, and then splice them back
into the tree by the consumer. The implementation can be found here:

https://github.com/microsoft/vscode-prompt-tsx/tree/connor4312/tools-api-v2

On the tool side, this looks like:

```ts
vscode.lm.registerTool('myTestTool', {
	async invoke(context, token): Promise<vscode.LanguageModelToolResult> {
		return {
			// context includes the token info:
			'mytype': await renderElementJSON(MyCustomPrompt, {}, context, token),
			toString() {
				return 'hello world!'
			}
		};
	},
});
```

I didn't make any nice wrappers yet, but the MVP consumer side looks like:

```
export class TestPrompt extends PromptElement {
    async render(_state: void, sizing: PromptSizing) {
        const result = await vscode.lm.invokeTool('myTestTool', {
			parameters: {},
			tokenBudget: sizing.tokenBudget,
            countTokens: (v, token) => tokenizer.countTokens(v, token),
		}, new vscode.CancellationTokenSource().token);

        return (
            <>
                <elementJSON data={result.mytype} />
            </>
        );
    }
}
```

I like this approach better. It avoids bleeding knowledge of TSX into
the extension host and comparatively simple.

* address comments

* address comments
This commit is contained in:
Connor Peet
2024-08-22 09:41:31 -07:00
committed by GitHub
parent bf52a5cfb2
commit 2bf25ee2fd
8 changed files with 130 additions and 33 deletions

View File

@@ -1489,7 +1489,7 @@ export function createApiFactoryAndRegisterActors(accessor: ServicesAccessor): I
checkProposedApiEnabled(extension, 'lmTools');
return extHostLanguageModelTools.registerTool(extension, toolId, tool);
},
invokeTool(toolId: string, parameters: Object, token: vscode.CancellationToken) {
invokeTool(toolId: string, parameters: vscode.LanguageModelToolInvokationOptions, token: vscode.CancellationToken) {
checkProposedApiEnabled(extension, 'lmTools');
return extHostLanguageModelTools.invokeTool(toolId, parameters, token);
},