WT UI Automation memory leak

[carlos-zamora]

Windows Terminal version

1.23.13503.0 (but issue was reported to be happening on earlier versions)

Windows build number

10.0.26543.0 (but issue was reported to be happening on earlier versions)

Other Software

Some kind of accessible tech like Narrator, NVDA, JAWS, etc...

Steps to reproduce

1. Open Terminal
2. Generate a lot of output (i.e. ls -Recurse)

Expected Behavior

Memory usage for Terminal should stay mostly the same. Maybe a minor increment.

Actual Behavior

Memory usage spikes up. Memory stays spiked if Narrator is closed. Looks like you have to close the tab to regain the memory.

[carlos-zamora]

@codeofdusk (along with a few others) had reported this to me in the past.

Did a deep dive with @lhecker in December to figure out what's going on. Here's some notes from that investigation:

• Issue: There's a memory leak in Windows Terminal when a screen reader is attached. Specifically, the generated ITextRangeProviders aren't destructed until a terminal session closes. From tracing the AddRef()/Release() calls, it looks like XAML is the one holding on to a reference to these text ranges and keeping them alive. We've confirmed that these text ranges are destroyed when the terminal session (i.e. tab, pane) closes. However, this means that they're adding up a lot over long sessions as new ranges are created with every character that is typed!
• WT UIA Structure:
  • The main automation providers used are ITextProvider and ITextRangeProvider. We create an ITextProvider for each terminal control (i.e. tab, pane), which then generates ITextRangeProviders to expose blocks of text. To reuse our UIA providers between Windows Terminal and Console Host, we have a ScreenInfoUiaProviderBase as the ITextProvider and UiaTextRangeBase as the ITextRangeProvider. The fully implemented UIA providers for Windows Terminal are TermControlUiaProvider (ITextProvider) and TermControlUiaTextRange (ITextRangeProvider).
  • However, Windows Terminal is written in XAML, so we needed to bridge the XAML accessibility classes to the UIA ones above. To do this, we added an InteractivityAutomationPeer (XAML ITextProvider) and a XamlUiaTextRange (XAML ITextRangeProvider). In the end, InteractivityAutomationPeer (XAML) calls functions out of TermControlUiaProvider (UIA) to generate TermControlUiaTextRanges (UIA), which are then wrapped in XamlUiaTextRanges (XAML) and returned.
  • The last piece here is that XAML creates the automation peer using TermControl::OnCreateAutomationPeer(). TermControl is just a terminal control that can be placed in a pane or tab. The output automation peer is the main way to interact with the UIA objects and extract text ranges.
• Notes from investigation:
  • Leonard managed to add a reference counter on XamlUiaTextRange and UiaTextRangeBase. We managed to find and fix some other memory leaks admittedly 😅, but we're still seeing that the objects aren't destructed until the terminal control is closed.
  • In our test scenario, we're simply typing into the terminal. This results in several GetSelection() and DocumentRange() calls, which makes sense because the automation client wants to detect the new position of the cursor as well as the contents of the terminal repeatedly.
  • Tracing the call into XAML's AutomationPeer implementation, we found that XAML is creating a reference for our object as well as a weak reference for it.

We've escalated this internally. Created an ADO item to track it internally too: MSFT 60670115

[codeofdusk]

CC @bramd, @Neurrone, @Simon818.