OllamaModelBackend.generate_from_raw silently swallows batch exceptions

[planetf1]

OllamaModelBackend.generate_from_raw silently swallows batch exceptions

Description

generate_from_raw uses asyncio.gather(..., return_exceptions=True) to run concurrent requests, but silently converts any exception into ModelOutputThunk(value=""), storing the error only in result._generate_log.extra["error"] — invisible to callers.

ollama.py:465-474

Impact

Callers have no way to detect failures. Tests asserting result.value is not None pass silently even when requests are failing, since empty string is not None.

Related

• Backends don't handle exceptions properly during generation #432 — streaming path exception handling (fixed by fix: do not post_process before finally in ModelOutputThunk.astream #580)
• Test issues (local, m1, 32GB) #573 — test flakiness where this manifested

[planetf1]

Proposed fix

Remove return_exceptions=True from asyncio.gather and delete the per-response exception handling block. Exceptions will then propagate naturally, consistent with how all other backends behave in generate_from_raw.

This is the simplest fix and aligns with the direction taken in #580 (fail fast, don't swallow exceptions).

Future consideration

Longer term, it may be worth considering explicit per-request error handling across all backends — e.g. a public error field on ModelOutputThunk — to support partial batch results. That's a broader API design question out of scope for this fix.

[planetf1]

Note: #599 tracks a related but distinct failure mode — Ollama returning genuinely empty responses under load (not exceptions). Both produce value="" but via different paths: this issue is the exception-swallowing code path; #599 is Ollama silently returning nothing. Worth keeping separate as the fixes are independent.