Skip to content

RFC-0027: Add route options size limit#1447

Open
rkoster wants to merge 3 commits intomainfrom
rfc-0027-route-options-size-limit
Open

RFC-0027: Add route options size limit#1447
rkoster wants to merge 3 commits intomainfrom
rfc-0027-route-options-size-limit

Conversation

@rkoster
Copy link
Contributor

@rkoster rkoster commented Mar 6, 2026

Summary

Adds a Size Limits section to RFC-0027 (Generic Per-Route Features) specifying that Cloud Controller must enforce a configurable maximum size for route options.

Changes

  • Default limit: 1024 bytes (1 KB)
  • Configurable via cc.max_route_options_size BOSH property
  • Cloud Controller returns HTTP 422 when limit is exceeded
  • Documents relationship with route emit interval for operator tuning

Context

This addresses feedback from the App-to-App mTLS RFC (PR #1438) where @maxmoehl raised concerns about NATS bandwidth impact of per-route options. The discussion concluded that a global size limit enforced at the Cloud Controller level (with immediate user feedback) is the appropriate solution.

Rationale

Route registration messages are transmitted via NATS at a configurable interval from each app instance to each GoRouter instance. Large route options can significantly impact NATS bandwidth in deployments with many app instances. A 1 KB default accommodates most use cases (e.g., ~25 app GUIDs in an mTLS allowlist) while protecting platform stability.

cc @maxmoehl

Add a new Size Limits section specifying that Cloud Controller must
enforce a configurable maximum size (default: 1 KB) for route options
to prevent excessive NATS bandwidth consumption.

- Default limit: 1024 bytes
- Configurable via cc.max_route_options_size BOSH property
- CC returns HTTP 422 when limit is exceeded
- Documents relationship with route emit interval for tuning

This addresses feedback from the App-to-App mTLS RFC (PR #1438) where
concerns were raised about NATS bandwidth impact of per-route options.
@beyhan beyhan requested review from a team, Gerg, beyhan, cweibel and stephanme and removed request for a team March 6, 2026 09:48
@beyhan beyhan added the toc label Mar 6, 2026
Copy link
Member

@maxmoehl maxmoehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for preparing this amendment. I like it overall, just 1.5 small comments.

rkoster and others added 2 commits March 6, 2026 15:57
Co-authored-by: Maximilian Moehl <44866320+maxmoehl@users.noreply.github.com>
- Remove duplicate 'Operator guidance' line from suggestion merge
- Change 'bandwidth usage' to 'resource usage' for consistency
- Add note that lowering size limit doesn't affect existing routes
@stephanme
Copy link
Member

We start the final comment period with the goal to accept this change to RFC-27 on 17.3.

@cweibel
Copy link

cweibel commented Mar 10, 2026

I like that this is controlled at the operator level

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Inbox

Development

Successfully merging this pull request may close these issues.

5 participants