[CLI] Running out of memory when running qwen3.5-0.8b-cuda-gpu:2 with Github Copilot

[ponnapz]

Problem

I was simply experimenting with trying to run some local models with GitHub copilot app.

I was able to do foundry run qwen3.5-0.8b-cuda-gpu:2 in CLI and try some queries, but trying to do the same in GitHub copilot app fails - after some seconds, suddenly both VRAM and RAM usage spikes and consumes entire available memory and session fails.

When I run the same model in Ollama and connect to GitHub Copilot app, that works as expected, so this looks like some issue specific to foundry local CLI and the Open AI compatible server?

Diagnostics

System

• OS: Microsoft Windows 10.0.26200
• Architecture: X64
• CPU: 13th Gen Intel(R) Core(TM) i7-13800H (20 logical cores)
• GPU: Intel Corporation Intel(R) Iris(R) Xe Graphics (2.0 GB), NVIDIA NVIDIA GeForce RTX 4060 Laptop GPU (4.0 GB)
• RAM: 9.5 GB available / 63.8 GB total
• Disk: 1.1 TB free / 1.4 TB total (D:)
• .NET: .NET 9.0.17

Service

• State: Ready
• Web URLs: http://127.0.0.1:65489
• CLI version: 0.10.1

Models

• Available variants: 161
• Loaded: 1
• Cached: 1

Connectivity

• Local service: Reachable
• Model registry: Reachable

Warnings

• No warnings.