fix(server): recover from panics in RPC and UI HTTP handlers by whoAbhishekSah · Pull Request #1877 · raystack/frontier

whoAbhishekSah · 2026-08-13T06:00:07Z

Problem

Frontier has no panic recovery in the request path. When a handler panics, the only net is Go's built-in per-connection recover in net/http:

The client's connection is dropped with no response at all (HTTP/1.1: empty reply; HTTP/2: INTERNAL_ERROR stream reset), instead of an RPC error.
The stack trace goes through net/http's error logger. With slog set as the default logger, that lands as a single INFO-level line with the whole stack stuffed into msg — invisible to anything watching for error-level logs, and with no request context.

Reproduced by planting a deliberate panic() in a handler:

$ curl -v .../FrontierService/ListAuthStrategies ...
* Empty reply from server            # HTTP/1.1
* HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

Fix

Pass connect.WithRecover(...) when building the Frontier and Admin service handlers. The recover callback logs the procedure, panic value, and stack through slog at error level, and returns CodeInternal with a generic message. It is placed before the interceptor chain so it sits outermost and also catches panics thrown from interceptors, not just handlers.
Wrap the UI server mux (/configs, /frontier-connect/ proxy, SPA routes) with an equivalent recovery handler that returns a plain 500, since WithRecover only covers connect handlers. It re-panics on http.ErrAbortHandler, matching net/http's own contract.
No panic details reach the caller in either path.

Out of scope: panics on internal background goroutines (audit log listener, shutdown watchers) — those are not HTTP calls and need their own recover treatment separately.

After the fix

Same deliberately panicking handler:

$ curl -s -w "\nHTTP %{http_code}\n" .../FrontierService/ListAuthStrategies ...
{"code":"internal","message":"internal server error"}
HTTP 500

and one structured log entry:

{"level":"ERROR","msg":"rpc handler panic","procedure":"/raystack.frontier.v1beta1.FrontierService/ListAuthStrategies","panic":"...","stack":"goroutine ..."}

Tests

A panicking connect handler returns HTTP 500 with "code":"internal", no panic detail in the body, and an error-level log entry carrying the procedure and stack.
The UI recovery wrapper returns a generic 500 on panic, passes normal requests through untouched, and re-panics on http.ErrAbortHandler.
Verified live against a running server over both HTTP/1.1 and HTTP/2 (output above).

🤖 Generated with Claude Code

A panicking handler previously fell through to net/http's per-connection recover: the client's connection was dropped with no response (HTTP/2 stream reset), and the stack trace bypassed structured logging. Add connect.WithRecover on the Frontier and Admin service handlers so a panic anywhere in the handler chain returns CodeInternal with a generic message, and wrap the UI server mux with an equivalent recovery handler that returns a plain 500. Both log the panic value and stack through slog at error level; no panic details reach the caller. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel · 2026-08-13T06:00:14Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
frontier	Ready	Preview	Aug 14, 2026 8:04am

coderabbitai · 2026-08-13T06:00:36Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8e712db5-e57d-424b-b9d7-6ea7457d47e9

📥 Commits

Reviewing files that changed from the base of the PR and between 838970f and f891372.

📒 Files selected for processing (1)

pkg/server/recovery_test.go

🚧 Files skipped from review as they are similar to previous changes (1)

pkg/server/recovery_test.go

📝 Walkthrough

Summary by CodeRabbit

Bug Fixes
- Prevented unexpected server crashes by recovering from panics in UI, RPC, and other HTTP request handlers.
- Returned safe internal-error responses without exposing sensitive panic details.
- Preserved standard behavior for normal requests and intentional connection-abort handling.
- Added diagnostic logging with relevant request context to support troubleshooting while protecting sensitive information.
- Ensured recovery behavior also covers health checks, webhooks, reflection, ping, and cross-origin requests.

Walkthrough

The server adds centralized panic recovery for Connect RPC and UI HTTP handlers. Recovery logs panic context, hides panic details from clients, returns controlled errors, and preserves http.ErrAbortHandler. Integration tests cover these behaviors.

Changes

Panic recovery

Layer / File(s)	Summary
Recovery wrappers and response handling `pkg/server/recovery.go`, `pkg/server/recovery_test.go`	Connect RPC panics return generic internal errors. UI panics return HTTP 500 responses when no response is committed. Committed responses preserve `http.ErrAbortHandler`. Tests cover logging, redaction, passthrough, response commitment, flushing, unwrapping, and `ReadFrom`.
Server handler integration `pkg/server/server.go`	The UI mux and Connect HTTP mux use `httpPanicRecovery`. Frontier and Admin ConnectRPC handlers use `connectPanicRecovery` as their outermost interceptor.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to f8913

The change adds localized panic recovery for RPC and UI HTTP handlers, returning generic errors while preserving structured server logging; no actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bda1a8bf-e75d-4b74-815a-5f79464ef6d5

📥 Commits

Reviewing files that changed from the base of the PR and between fafc986 and f5b2cae.

📒 Files selected for processing (3)

pkg/server/recovery.go
pkg/server/recovery_test.go
pkg/server/server.go

…covery Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

coveralls · 2026-08-13T06:06:32Z

Coverage Report for CI Build 31782410231

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage increased (+0.1%) to 48.323%

Details

Coverage increased (+0.1%) from the base build.
Patch coverage: 3 uncovered changes across 2 files (49 of 52 lines covered, 94.23%).
126 coverage regressions across 4 files.

Uncovered Changes

File	Changed	Covered	%
pkg/server/recovery.go	46	44	95.65%
pkg/server/server.go	6	5	83.33%

Coverage Regressions

126 previously-covered lines in 4 files lost coverage.

File	Lines Losing Coverage	Coverage
internal/store/postgres/organization_repository.go	66	79.25%
internal/store/spicedb/relation_repository.go	31	0.0%
pkg/db/db.go	26	24.39%
config/config.go	3	83.33%

Coverage Stats


Relevant Lines:	39809
Covered Lines:	19237
Line Coverage:	48.32%
Coverage Strength:	15.42 hits per line

💛 - Coveralls

…body When a UI handler panics after writing part of a response, the status line is already on the wire: http.Error cannot change it and would only append the error text to the partial body, so the client would read a corrupt 200. Track response commitment; once committed, log the panic and re-panic with http.ErrAbortHandler so net/http drops the connection and the client sees a truncated response instead of a fake success. The wrapper forwards Flush for the reverse proxy's streaming. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Address review feedback on the recovery wrapper and its coverage: - Add Unwrap on committedWriter so http.ResponseController can reach the underlying writer's Hijacker, deadline, and full-duplex methods; without it a protocol upgrade through the connect reverse proxy on the UI server would fail. - Add ReadFrom passthrough so copies into the wrapper keep the underlying writer's optimized path, which io.Copy looks up on the destination directly without walking Unwrap. - Wrap the connect server mux with the same recovery handler. The webhook bridge does its own parsing before the protected handler, and ping, health, reflection, and CORS had no recovery at all. RPC panics are still converted to connect error codes by WithRecover first; the outer wrapper only sees panics that escape it. - Rename uiPanicRecovery to httpPanicRecovery since it now fronts both servers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

coderabbitai

Actionable comments posted: 1

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 03e7e87a-e04d-437e-84ce-32e176594906

📥 Commits

Reviewing files that changed from the base of the PR and between eecc659 and 838970f.

📒 Files selected for processing (3)

pkg/server/recovery.go
pkg/server/recovery_test.go
pkg/server/server.go

🚧 Files skipped from review as they are similar to previous changes (1)

pkg/server/recovery.go

io.Copy prefers the source's WriteTo over the destination's ReadFrom, and strings.Reader implements WriteTo, so the test never reached ReadFrom. Call it directly so the test exercises the method it checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel Bot deployed to Preview August 13, 2026 06:00 View deployment

coderabbitai Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread pkg/server/recovery.go Outdated

refactor(server): reuse pkg/errors ErrInternalServerError in panic re…

eecc659

…covery Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vercel Bot deployed to Preview August 13, 2026 06:05 View deployment

whoAbhishekSah requested a review from AmanGIT07 August 13, 2026 06:12

vercel Bot deployed to Preview August 13, 2026 06:17 View deployment

rohilsurana reviewed Aug 13, 2026

View reviewed changes

Comment thread pkg/server/recovery.go

Comment thread pkg/server/recovery.go

AmanGIT07 reviewed Aug 13, 2026

View reviewed changes

Comment thread pkg/server/server.go Outdated

vercel Bot deployed to Preview August 13, 2026 10:21 View deployment

coderabbitai Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread pkg/server/recovery_test.go Outdated

AmanGIT07 approved these changes Aug 13, 2026

View reviewed changes

vercel Bot deployed to Preview August 14, 2026 08:04 View deployment

whoAbhishekSah merged commit 5bacc1b into main Aug 14, 2026
8 checks passed

whoAbhishekSah deleted the fix/http-handler-panic-recovery branch August 14, 2026 08:41

Navigation Menu

Uh oh!

fix(server): recover from panics in RPC and UI HTTP handlers - #1877

fix(server): recover from panics in RPC and UI HTTP handlers#1877
whoAbhishekSah merged 5 commits into
mainfrom
fix/http-handler-panic-recovery

whoAbhishekSah commented Aug 13, 2026

Uh oh!

vercel Bot commented Aug 13, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 13, 2026 •

edited

Loading

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coveralls commented Aug 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

whoAbhishekSah commented Aug 13, 2026

Problem

Fix

After the fix

Tests

Uh oh!

vercel Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coveralls commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 31782410231

Coverage increased (+0.1%) to 48.323%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vercel Bot commented Aug 13, 2026 •

edited

Loading

coderabbitai Bot commented Aug 13, 2026 •

edited

Loading

coveralls commented Aug 13, 2026 •

edited

Loading