Summary
On a self-hosted install (no Robusta UI/account), a customPlaybooks ask_holmes action fails because robusta-runner POSTs to /api/investigate,
but the bundled Holmes server only exposes /api/chat. Automated AI analysis of alerts never runs.
Environment
- Robusta Helm chart: 0.41.0 (also confirmed present in 0.42.0)
- robusta-runner image:
robustadev/robusta-runner:0.41.0
- holmes image:
robustadev/holmes:0.32.0 (0.42.0 ships 0.33.0)
- Self-hosted, Azure OpenAI (
MODEL=azure/...), enableHolmesGPT: true, no Robusta UI sink / account
- Sink: MS Teams
What happens
A customPlaybook ask_holmes action on on_pod_crash_loop fails in robusta-runner:
ERROR Holmes responded 404 on POST http://robusta-holmes.robusta.svc.cluster.local:80/api/investigate: {"detail":"Not Found"}
robusta.utils.error_codes.ActionException: Holmes internal configuration error.
File ".../core/playbooks/internal/ai_integration.py", line 136, in ask_holmes
Root cause
robusta-runner ai_integration.py (0.41 and current master) POSTs to /api/investigate and /api/stream/investigate.
- The bundled Holmes server (
0.32.0, and master) only registers: /api/chat, /api/model, /api/oauth/callback, /healthz, /readyz — confirmed
via the pod's own /openapi.json.
/api/investigate existed in Holmes ≤ 0.13.0 and was removed in the server rewrite; the runner was never migrated to /api/chat.
So the shipped chart pairs a runner and a Holmes that disagree on the endpoint, and self-hosted automated investigation is broken out of the box.
Expected
Self-hosted automated alert investigation (documented as supported with OpenAI/Azure/Bedrock) should work, or robusta-runner should call /api/chat.
Reproduce
- Install robusta chart 0.41/0.42 self-hosted,
enableHolmesGPT: true, Azure model configured.
- Add a
customPlaybook with an ask_holmes action (e.g. on on_pod_crash_loop).
- Trigger a CrashLoopBackOff pod → runner logs the 404 above; the Teams/Slack alert arrives with no AI section.
Notes (other gotchas found while debugging, in case docs need updating)
- Model env must be under
holmes.additionalEnvVars (a holmes.env: key is silently ignored → Holmes pod has no model).
ask_holmes also requires investigation_type (e.g. issue) and a non-null context (context.issue_type), or it raises AttributeError: 'NoneType' object has no attribute 'get' in build_investigation_title.
Summary
On a self-hosted install (no Robusta UI/account), a
customPlaybooksask_holmesaction fails because robusta-runner POSTs to/api/investigate,but the bundled Holmes server only exposes
/api/chat. Automated AI analysis of alerts never runs.Environment
robustadev/robusta-runner:0.41.0robustadev/holmes:0.32.0(0.42.0 ships0.33.0)MODEL=azure/...),enableHolmesGPT: true, no Robusta UI sink / accountWhat happens
A
customPlaybookask_holmesaction onon_pod_crash_loopfails in robusta-runner:Root cause
robusta-runnerai_integration.py(0.41 and currentmaster) POSTs to/api/investigateand/api/stream/investigate.0.32.0, andmaster) only registers:/api/chat,/api/model,/api/oauth/callback,/healthz,/readyz— confirmedvia the pod's own
/openapi.json./api/investigateexisted in Holmes ≤0.13.0and was removed in the server rewrite; the runner was never migrated to/api/chat.So the shipped chart pairs a runner and a Holmes that disagree on the endpoint, and self-hosted automated investigation is broken out of the box.
Expected
Self-hosted automated alert investigation (documented as supported with OpenAI/Azure/Bedrock) should work, or robusta-runner should call
/api/chat.Reproduce
enableHolmesGPT: true, Azure model configured.customPlaybookwith anask_holmesaction (e.g. onon_pod_crash_loop).Notes (other gotchas found while debugging, in case docs need updating)
holmes.additionalEnvVars(aholmes.env:key is silently ignored → Holmes pod has no model).ask_holmesalso requiresinvestigation_type(e.g.issue) and a non-nullcontext(context.issue_type), or it raisesAttributeError: 'NoneType' object has no attribute 'get'inbuild_investigation_title.