chat: store disk sessions using an 'append log'

[connor4312]

This switches from a big blob of JSON to an append-only log of operations for the chat session. Sending a new request looks something like this:

{"kind":2,"k":["requests"],"v":[{"requestId":"request_125e2fa9-1cc2-4197-b023-580eabc9e229","timestamp":1767918567679,"message":{"parts":[{"range":{"start":0,"endExclusive":32},"editorRange":{"startLineNumber":1,"startColumn":1,"endLineNumber":1,"endColumn":33},"text":"what kinds of things can you do?","kind":"text"}],"text":"what kinds of things can you do?"},"agent":{"extensionId":{"value":"GitHub.copilot-chat","_lower":"github.copilot-chat"},"extensionVersion":"0.35.2025120903","publisherDisplayName":"GitHub","extensionPublisherId":"GitHub","extensionDisplayName":"GitHub Copilot Chat","id":"github.copilot.editsAgent","description":"Edit files in your workspace in agent mode","when":"config.chat.agent.enabled","metadata":{"themeIcon":{"id":"tools"},"hasFollowups":false,"supportIssueReporting":true,"additionalWelcomeMessage":{"value":"If handling customer data, [disable telemetry](command:workbench.action.openSettings?%5B%22telemetry.telemetryLevel%22%5D).","isTrusted":{"enabledCommands":["workbench.action.openSettings"]},"supportThemeIcons":false,"supportHtml":false,"supportAlertSyntax":false,"uris":{"command:workbench.action.openSettings?%5B%22telemetry.telemetryLevel%22%5D":{"$mid":1,"path":"workbench.action.openSettings","scheme":"command","query":"[\"telemetry.telemetryLevel\"]"}}}},"name":"agent","fullName":"GitHub Copilot","isDefault":true,"locations":["panel"],"modes":["agent"],"slashCommands":[{"name":"list"}],"disambiguation":[]},"modelId":"copilot/gpt-4o","variableData":{"variables":[{"value":"Repository name: ext-apps\nOwner: modelcontextprotocol\nCurrent branch: main\nDefault branch: main","name":"modelcontextprotocol/ext-apps","modelDescription":"Information about the current repository. You can use this information when you need to calculate diffs or compare changes with the default branch","id":"modelcontextprotocol/ext-apps","kind":"workspace"},{"id":"vscode.prompt.instructions.text","name":"prompt:instructionsList","value":"<instructions>\nHere is a list of instruction files that contain rules for modifying or creating new code.\nThese files are important for ensuring that the code is modified or created correctly.\nPlease make sure to follow the rules specified in these files when working with the codebase.\nIf the file is not already available as attachment, use the #tool:readFile tool to acquire it.\nMake sure to acquire the instructions before making any changes to the code.\n<instruction>\n<description>**CRITICAL**: Read this file FIRST before answering ANY Azure-related questions or performing ANY Azure operations. This file must be loaded as context whenever Azure is explicitly mentioned. It provides accurate and up-to-date information about Azure. **MUST** be read when user mentions: Azure, Azure Functions, AWS Lambda to Azure Functions migration, Azure Static Web App, or any specific Azure service name. Load this instruction file ONLY for Azure-related requests, NOT for generic cloud or deployment questions.</description>\n<file>/Users/connor/.vscode-oss-dev/extensions/ms-azuretools.vscode-azure-github-copilot-1.0.153-darwin-arm64/resources/azureRules/azure.instructions.md</file>\n</instruction>\n</instructions>\n\n","kind":"promptText","modelDescription":"Prompt instructions list","automaticallyAdded":true,"toolReferences":[{"kind":"tool","id":"copilot_readFile","name":"Read File","range":{"start":347,"endExclusive":361}}]}]},"response":[{"kind":"mcpServersStarting","didStartServerIds":[]}],"responseId":"response_8101b2ad-66f7-4e58-ae91-81ff032192e6","modelState":0,"contentReferences":[],"codeCitations":[],"timeSpentWaiting":1767918567679}]}
{"kind":2,"k":["requests",5,"response"],"v":[{"value":"I can help you with a wide range of tasks, including:\n\n1. **Code Assistance**: Writing, debugging, and optimizing code in various languages and frameworks.\n2. **File Management**: Creating, editing, and organizing files in your project.\n3. **Project Setup**: Setting up new projects, configuring tools, and managing dependencies.\n4. **Error Fixing**: Identifying and resolving errors in your code.\n5. **Code Refactoring**: Improving code readability, structure, and performance.\n6. **Documentation**: Generating or improving documentation for your code.\n7. **Version Control**: Assisting with Git operations like commits, branches, and diffs.\n8. **Task Automation**: Writing scripts or setting up tasks to automate repetitive processes.\n9. **Learning and Research**: Explaining concepts, providing examples, or researching solutions.\n10. **Integration**: Connecting APIs, libraries, or services into your project.\n\nLet me know what you need, and I'll get started!","supportThemeIcons":false,"supportHtml":false,"baseUri":{"$mid":1,"path":"/Users/connor/Github/mcp-ext-apps/examples/basic-server-react/","scheme":"file"}}]}
{"kind":1,"k":["requests",5,"result"],"v":{"timings":{"firstProgress":2058,"totalElapsed":3838},"metadata":{"codeBlocks":[],"renderedUserMessage":[{"type":1,"text":"<attachments>\n<attachment id=\"modelcontextprotocol/ext-apps\">\nInformation about the current repository. You can use this information when you need to calculate diffs or compare changes with the default branch:\nRepository name: ext-apps\nOwner: modelcontextprotocol\nCurrent branch: main\nDefault branch: main\n</attachment>\n\n</attachments>\n<context>\nThe current date is January 8, 2026.\nTerminals:\nTerminal: zsh\n\n</context>\n<editorContext>\nThe user's current file is /Users/connor/Github/mcp-ext-apps/examples/basic-server-react/.vscode/mcp.json. \n</editorContext>\n<reminderInstructions>\nYou are an agent - you must keep going until the user's query is completely resolved, before ending your turn and yielding back to the user. ONLY terminate your turn when you are sure that the problem is solved, or you absolutely cannot continue.\nYou take action when possible- the user is expecting YOU to take action and go to work for them. Don't ask unnecessary questions about the details if you can simply DO something useful instead.\nWhen using the insert_edit_into_file tool, avoid repeating existing code, instead use a line comment with \\`...existing code...\\` to represent regions of unchanged code.\n\n</reminderInstructions>\n<userRequest>\nwhat kinds of things can you do?\n</userRequest>\n"},{"type":3,"cacheType":"ephemeral"}],"toolCallRounds":[{"response":"I can help you with a wide range of tasks, including:\n\n1. **Code Assistance**: Writing, debugging, and optimizing code in various languages and frameworks.\n2. **File Management**: Creating, editing, and organizing files in your project.\n3. **Project Setup**: Setting up new projects, configuring tools, and managing dependencies.\n4. **Error Fixing**: Identifying and resolving errors in your code.\n5. **Code Refactoring**: Improving code readability, structure, and performance.\n6. **Documentation**: Generating or improving documentation for your code.\n7. **Version Control**: Assisting with Git operations like commits, branches, and diffs.\n8. **Task Automation**: Writing scripts or setting up tasks to automate repetitive processes.\n9. **Learning and Research**: Explaining concepts, providing examples, or researching solutions.\n10. **Integration**: Connecting APIs, libraries, or services into your project.\n\nLet me know what you need, and I'll get started!","toolCalls":[],"toolInputRetry":0,"id":"e6ff2540-af2b-4fdc-8862-183b6c25c3e9"}],"modelMessageId":"a33d84e3-dbdf-4f75-95dd-bfd96a4cbec2","responseId":"456849bf-a8ec-4149-b76a-b0ddaa1d99ea","sessionId":"53bdd40b-39bd-413d-8069-5562488888a4","agentId":"github.copilot.editsAgent"},"details":"GPT-4o • 0x"}}
{"kind":1,"k":["requests",5,"responseMarkdownInfo"],"v":[]}
{"kind":1,"k":["requests",5,"followups"],"v":[]}
{"kind":1,"k":["requests",5,"modelState"],"v":1}

Essentially in this PR there are three parts:

1. Defining the Adapt in chatSessionOperationLog.ts that defines types that can translate to/from objects to the on disk state. The diffing in there was all written by Opus and I'm going to do some more validation before merging.

2. Making the requestSchema. Our models atm are very mutable. This logic describes how to compare the model objects. It's essentially a customized deep object equality that we can customize to avoid having to compare every single property of every object each time (especially with big tool outputs and images)

   This is all type-safe, both for reading the request models and creating the ISerializableChatData. I do want to simplify it, I feel like this might be a bit verbose in the first cut, but one thing this does do very well which I felt was important was forcing developers to think about serialization whenever a new response type is added, or whenever a new property is added to serialized data.

3. Swapping out to use that log in our logic. The log is a bit stateful so we pass it around and keep it on the model to re-use next time it's saved.

4. Adding an appendFile operation on the IFileService/IFileSystemProvider. We previously did not have any way to rewrite part of a file. @bpasero please review 😅

Closes #285251

[Copilot]

Pull request overview

This PR replaces the JSON blob storage for chat sessions with an append-only operation log to improve performance for long chat sessions. The implementation includes:

1. A new Adapt framework in chatSessionOperationLog.ts that defines how to serialize chat models and detect changes
2. An appendFile operation added to IFileService and all file system providers
3. Integration with ChatModel and ChatSessionStore to use the new serialization approach

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
chatSessionOperationLog.ts	New file implementing the Adapt framework and LogAdapter for incremental chat serialization
chatSessionOperationLog.test.ts	Comprehensive test coverage for the new operation log functionality
files.ts	Added FileAppend capability and IFileSystemProviderWithFileAppendCapability interface
fileService.ts	Implemented appendFile method with native support and read-then-write fallback
diskFileSystemProvider.ts	Native fs.appendFile implementation for Node.js
indexedDBFileSystemProvider.ts	Append implementation for browser IndexedDB
inMemoryFilesystemProvider.ts	Append implementation for in-memory testing
diskFileSystemProviderClient.ts	IPC client support for appendFile
diskFileSystemProviderServer.ts	IPC server support for appendFile
chatModel.ts	Updated types to support serialization references and removed lastMessageDate field
chatSessionStore.ts	Integrated operation log for incremental writes
chatService/chatService.ts	Added toJSON requirements to IChatToolInvocation, IChatTask, IChatElicitationRequest
chatServiceImpl.ts	Updated to use ISerializedChatDataReference
chatViewModel.ts	Removed avatarIcon property
editingService.ts	Added toJSON implementation for IChatMultiDiffData
Test files	Updated to match new type signatures

Comments suppressed due to low confidence (3)

src/vs/workbench/contrib/chat/common/chatService/chatService.ts:347

• The interface IChatElicitationRequest now requires a toJSON method, but no implementation is provided. Any class implementing IChatElicitationRequest will need to provide this method or will fail to compile.

	toJSON(): IChatElicitationRequestSerialized;
}

src/vs/workbench/contrib/chat/common/model/chatSessionOperationLog.ts:593

• Similar issue with the comment - "deprecated, modelState is used inste, Adapt.constant(a)d" appears to be corrupted. Should be clarified to explain why isCanceled is deprecated and what replaces it.

	isCanceled: Adapt.t(() => undefined, Adapt.primitive()), // deprecated, modelState is used inste, Adapt.constant(a)d

src/vs/workbench/contrib/chat/common/chatService/chatService.ts:230

• The interface IChatTask now requires a toJSON method, but no implementation is provided. Any class implementing IChatTask will need to provide this method or will fail to compile.

	toJSON(): IChatTaskSerialized;
}

[bpasero]

There is:

• comments from Copilot that are possibly relevant
• missing tests for the added method on the disk file system provider node.js impl
• red CI

Will review once the above is settled.

[connor4312]

@bpasero @roblourens should be ready for review now. I also added a (hidden) setting in case people run into issues and need to bail out, or for comparative analysis.

[bpasero]

@connor4312 CI seems Red still

[bpasero]

Sounds like this method deserves a test.

[bpasero]

To preserve as much of the logic we have around locking, can we use the open, write, close methods? It seems there is a mode a that can be set to append?

[bpasero]

I wonder if it wouldn't be just easier to have another option in IWriteFileOptions to append? Then you could pretty much reuse everything all the way down to the implementation and just use the a mode when writing?