Using ollama models results crush getting stuck in "waiting state"

[puresick]

Description

When using Crush with any ollama model configured, sending a prompt results in crush being stuck in its "waiting/generating" state:

I can observe that my GPU is being used, i.e. that the model runs for some time, but that it stops at one point. So something is being calculated, but crush stays in that UI state shown above forever.

I can confirm that the ollama and the models (using ministral-3:8b and gpt-oss:20b here) are working by using ollama CLI and open web UI. Using the openai-compat type does work with Ministrals Codestral cloud API.

Here is my crush configuration:

{
  "$schema": "https://charm.land/crush.json",
  "options": {
    "disable_metrics": true
  },
  "providers": {
    "codestral": {
      "type": "openai-compat",
      "base_url": "https://codestral.mistral.ai/v1/",
      "api_key": "REDACTED",
      "models": [
        {
          "id": "codestral-latest",
          "name": "codestral-latest",
          "cost_per_1m_in": 0,
          "cost_per_1m_out": 0,
          "cost_per_1m_in_cached": 0,
          "cost_per_1m_out_cached": 0,
          "context_window": 32000,
          "default_max_tokens": 5000
        }
      ]
    },
    "ollama": {
      "name": "Ollama",
      "type": "openai-compat",
      "base_url": "http://localhost:11434/v1/",
      "models": [
        {
          "id": "ministral-3:8b",
          "name": "Ministral 3 8b",
          "context_window": 32000,
          "default_max_tokens": 4000
        },
        {
          "id": "gpt-oss:20b",
          "name": "gpt-oss:20b",
          "cost_per_1m_in": 0,
          "cost_per_1m_out": 0,
          "cost_per_1m_in_cached": 0,
          "cost_per_1m_out_cached": 0,
          "context_window": 128000,
          "default_max_tokens": 16000
        }
      ]
    }
  }
}

Version

0.30.2

Environment

Arch Linux, fish shell, foot terminal, tmux, ollama 0.13.5

[Gustave-241021]

@puresick You can pull this branch to see if Ollama works properly

This seems to be a known issue with Ollama. Ollama cannot stream responses when tool calls are enabled – when a request includes the tools parameter, Ollama buffers the entire response and sends it all at once, instead of streaming it gradually. This is why Crush gets stuck in a waiting state

When disable_tools: true is enabled:

• The agent cannot use any tools (bash, edit, view, grep, etc.)
• Only text/reasoning responses are available
• This is a trade-off to enable streaming with problematic providers

[Gustave-241021]

I am checking on the fantasy side to see if this issue can be fixed.

[puresick]

@Gustave-241021 Thanks for looking into this.
Unfortunately without tool calling crush loses a good chunk of its usefulness.

If I understand correctly, charmbracelet/fantasy#113 would be closer to a proper fix with tool support, right?

[Gustave-241021]

@Gustave-241021 Thanks for looking into this. Unfortunately without tool calling crush loses a good chunk of its usefulness.

If I understand correctly, charmbracelet/fantasy#113 would be closer to a proper fix with tool support, right?

yes

[Gustave-241021]

@puresick You can pull the dependencies I submitted and import them into Crush to check if the issue has been fixed

I am always here to listen to your feedback and be prepared to make adjustments

[andreynering]

Hey @puresick 👋

@Gustave-241021 fixed this and I'm going to release the fix in minutes.

Thank you both!