Skip to content

feat: allow L4 upstream schemes (tcp/tls/udp) for stream backends - #449

Open
AlinsRan wants to merge 1 commit into
masterfrom
ee-2803-l4-upstream-scheme
Open

feat: allow L4 upstream schemes (tcp/tls/udp) for stream backends#449
AlinsRan wants to merge 1 commit into
masterfrom
ee-2803-l4-upstream-scheme

Conversation

@AlinsRan

@AlinsRan AlinsRan commented Jul 29, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Description

Sync of apache/apisix-ingress-controller#2830 (fixes apache/apisix-ingress-controller#2803).

APISIX accepts tcp, tls and udp as upstream schemes for L4 proxying, and scheme: tls is what makes the gateway establish the TLS session with the upstream for a stream route (plain TCP in, TLS out).

Both ApisixUpstream.spec.scheme and BackendTrafficPolicy.spec.scheme restricted the value to the L7 set (http|https|grpc|grpcs), so this could only be configured through the Admin API.

The translation path already carries the scheme into the generated upstream for both entry points:

  • ApisixRoute stream backends merge the ApisixUpstream named after the backend Service;
  • Gateway API TCPRoute backends take the scheme from BackendTrafficPolicy.

So only CRD validation needed relaxing.

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: my-service     # same name as the backend Service
spec:
  scheme: tls

Tests

  • unit: TestTranslateApisixRouteStreamUpstreamScheme, TestTranslateTCPRouteUpstreamScheme
  • e2e: Test ApisixRoute With StreamRoute / TCP Proxy with TLS upstream and TCPRoute E2E Test / TCPRoute With BackendTrafficPolicy — both send plain HTTP over a TCP listener to nginx's TLS port and assert 200 + X-Port: 443, which only passes if the gateway performs the TLS handshake with the upstream.

Known follow-ups (not in this PR)

  • BackendTrafficPolicy.spec.scheme keeps +kubebuilder:default=http, so a policy attached only for load balancing still writes scheme: http onto an L4 upstream.
  • SchemeToPort has no entry for tls, so an externalNodes/ExternalName upstream without an explicit port defaults to 80.
  • portLevelSettings resets the top-level scheme for ports that do not set it.

Summary by CodeRabbit

  • New Features
    • Added support for Layer 4 upstream schemes: tcp, tls, and udp.
    • Expanded allowed scheme values for relevant CRDs so L4 can be used with stream-based routing.
  • Documentation
    • Updated scheme field descriptions to clearly distinguish L7 (http, https, grpc, grpcs) vs L4 (tcp, tls, udp) behavior and limitations for HTTP routes.
  • Tests
    • Added/expanded test coverage to validate upstream scheme handling for TCP/TLS/UDP scenarios, including end-to-end stream routing.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 876a2118-4fe7-4f61-bb19-6848638d595e

📥 Commits

Reviewing files that changed from the base of the PR and between 075d13c and 6e34fbf.

📒 Files selected for processing (10)
  • api/v1alpha1/backendtrafficpolicy_types.go
  • api/v2/apisixupstream_types.go
  • config/crd/bases/apisix.apache.org_apisixupstreams.yaml
  • config/crd/bases/apisix.apache.org_backendtrafficpolicies.yaml
  • docs/en/latest/reference/api-reference.md
  • internal/adc/translator/apisixroute_test.go
  • internal/adc/translator/l4route_test.go
  • internal/adc/translator/tcproute.go
  • test/e2e/crds/v2/streamroute.go
  • test/e2e/gatewayapi/tcproute.go
💤 Files with no reviewable changes (1)
  • internal/adc/translator/tcproute.go
🚧 Files skipped from review as they are similar to previous changes (9)
  • api/v2/apisixupstream_types.go
  • config/crd/bases/apisix.apache.org_backendtrafficpolicies.yaml
  • api/v1alpha1/backendtrafficpolicy_types.go
  • test/e2e/gatewayapi/tcproute.go
  • config/crd/bases/apisix.apache.org_apisixupstreams.yaml
  • test/e2e/crds/v2/streamroute.go
  • internal/adc/translator/apisixroute_test.go
  • internal/adc/translator/l4route_test.go
  • docs/en/latest/reference/api-reference.md
📝 Walkthrough

Walkthrough

The change expands upstream scheme validation and documentation to support tcp, tls, and udp, adds translator coverage for stream schemes, and adds end-to-end tests for TLS upstream routing through ApisixRoute and Gateway API TCPRoute.

Changes

L4 upstream scheme support

Layer / File(s) Summary
Scheme contracts and documentation
api/..., config/crd/bases/..., docs/en/latest/reference/api-reference.md		 Scheme validation and descriptions now include tcp, tls, and udp for L4 proxy usage.
Stream translation coverage
internal/adc/translator/...		 Unit tests verify configured schemes and endpoint hosts for ApisixRoute and TCPRoute stream backends.
End-to-end TLS stream validation
test/e2e/crds/v2/streamroute.go, test/e2e/gatewayapi/tcproute.go		 End-to-end tests validate TLS upstream routing through ApisixRoute and Gateway API TCPRoute.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: nic-6443, shreemaan-abhishek

Sequence Diagram(s)

sequenceDiagram
  participant Route as ApisixRoute or TCPRoute
  participant Translator as Stream translator
  participant APISIX as APISIX upstream
  participant Backend as TLS backend
  Route->>Translator: Provide stream backend with scheme tls
  Translator->>APISIX: Configure scheme and endpoint host
  APISIX->>Backend: Open TLS connection
  Backend-->>APISIX: Return upstream response
Loading
🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
E2e Test Quality Review ⚠️ Warning The new e2e tests hit real services, but the CRD-based upstream test skips the usual acceptance wait, so it can race reconciliation; invalid-scheme coverage is also missing. Use the existing acceptance helpers (MustApplyAPIv2/ResourceApplied) for ApisixUpstream and BackendTrafficPolicy, and add one negative validation test for an invalid scheme.
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: enabling L4 upstream schemes for stream backends.
Linked Issues check ✅ Passed The PR satisfies #2803 by allowing scheme tls in CRDs and propagating it for stream backends, with unit and e2e coverage.
Out of Scope Changes check ✅ Passed The changes stay focused on L4 scheme support, schema updates, tests, and docs, with no clear unrelated additions.
Security Check ✅ Passed Only CRD enum/docs/test updates; no secret handling, auth, ownership, or TLS-flag logic changes were introduced.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ee-2803-l4-upstream-scheme

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

conformance test report - apisix mode

apiVersion: gateway.networking.k8s.io/v1
date: "2026-07-29T08:06:44Z"
gatewayAPIChannel: experimental
gatewayAPIVersion: v1.6.0
implementation:
  contact:
  - https://github.com/apache/apisix-ingress-controller/issues
  organization: APISIX
  project: apisix-ingress-controller
  url: https://github.com/apache/apisix-ingress-controller.git
  version: v2.0.0
kind: ConformanceReport
mode: default
profiles:
- core:
    result: partial
    skippedTests:
    - HTTPRouteHTTPSListener
    - HTTPRouteInvalidBackendRefUnknownKind
    - HTTPRouteInvalidCrossNamespaceBackendRef
    - HTTPRouteInvalidNonExistentBackendRef
    - HTTPRouteListenerHostnameMatching
    - HTTPRouteMultipleGateways
    - HTTPRouteNoBackendRefs
    statistics:
      Failed: 0
      Passed: 30
      Skipped: 7
  extended:
    result: partial
    skippedTests:
    - HTTPRouteRedirectPortAndScheme
    statistics:
      Failed: 0
      Passed: 12
      Skipped: 1
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    - HTTPRouteBackendProtocolWebSocket
    - HTTPRouteDestinationPortMatching
    - HTTPRouteHostRewrite
    - HTTPRouteMethodMatching
    - HTTPRoutePathRewrite
    - HTTPRoutePortRedirect
    - HTTPRouteQueryParamMatching
    - HTTPRouteRequestMirror
    - HTTPRouteResponseHeaderModification
    - HTTPRouteSchemeRedirect
    unsupportedFeatures:
    - BackendTLSPolicy
    - BackendTLSPolicySANValidation
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - HTTPRoute303RedirectStatusCode
    - HTTPRoute307RedirectStatusCode
    - HTTPRoute308RedirectStatusCode
    - HTTPRouteBackendProtocolH2C
    - HTTPRouteBackendRequestHeaderModification
    - HTTPRouteBackendTimeout
    - HTTPRouteCORS
    - HTTPRouteNamedRouteRule
    - HTTPRouteParentRefPort
    - HTTPRoutePathRedirect
    - HTTPRouteRequestMultipleMirrors
    - HTTPRouteRequestPercentageMirror
    - HTTPRouteRequestTimeout
    - HTTPRouteRetry
    - HTTPRouteRetryBackendTimeout
    - HTTPRouteRetryConnectionError
    - ListenerSet
  name: GATEWAY-HTTP
  summary: Core tests partially succeeded with 7 test skips. Extended tests partially
    succeeded with 1 test skips.
- core:
    result: partial
    skippedTests:
    - GRPCRouteListenerHostnameMatching
    statistics:
      Failed: 0
      Passed: 14
      Skipped: 1
  extended:
    result: success
    statistics:
      Failed: 0
      Passed: 1
      Skipped: 0
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    unsupportedFeatures:
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - ListenerSet
  name: GATEWAY-GRPC
  summary: Core tests partially succeeded with 1 test skips. Extended tests succeeded.
- core:
    result: partial
    skippedTests:
    - TLSRouteHostnameIntersection
    - TLSRouteInvalidBackendRefNonexistent
    - TLSRouteInvalidBackendRefUnknownKind
    - TLSRouteSimpleSameNamespace
    statistics:
      Failed: 0
      Passed: 16
      Skipped: 4
  extended:
    result: partial
    skippedTests:
    - TLSRouteTerminateSimpleSameNamespace
    statistics:
      Failed: 0
      Passed: 3
      Skipped: 1
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    - TLSRouteModeTerminate
    unsupportedFeatures:
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - ListenerSet
    - TLSRouteModeMixed
  name: GATEWAY-TLS
  summary: Core tests partially succeeded with 4 test skips. Extended tests partially
    succeeded with 1 test skips.
succeededProvisionalTests:
- GatewayOptionalAddressValue

@github-actions

github-actions Bot commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

conformance test report - apisix-standalone mode

apiVersion: gateway.networking.k8s.io/v1
date: "2026-07-29T08:07:03Z"
gatewayAPIChannel: experimental
gatewayAPIVersion: v1.6.0
implementation:
  contact:
  - https://github.com/apache/apisix-ingress-controller/issues
  organization: APISIX
  project: apisix-ingress-controller
  url: https://github.com/apache/apisix-ingress-controller.git
  version: v2.0.0
kind: ConformanceReport
mode: default
profiles:
- core:
    result: partial
    skippedTests:
    - HTTPRouteHTTPSListener
    - HTTPRouteInvalidBackendRefUnknownKind
    - HTTPRouteInvalidCrossNamespaceBackendRef
    - HTTPRouteInvalidNonExistentBackendRef
    - HTTPRouteListenerHostnameMatching
    - HTTPRouteMultipleGateways
    - HTTPRouteNoBackendRefs
    statistics:
      Failed: 0
      Passed: 30
      Skipped: 7
  extended:
    result: partial
    skippedTests:
    - HTTPRouteRedirectPortAndScheme
    statistics:
      Failed: 0
      Passed: 12
      Skipped: 1
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    - HTTPRouteBackendProtocolWebSocket
    - HTTPRouteDestinationPortMatching
    - HTTPRouteHostRewrite
    - HTTPRouteMethodMatching
    - HTTPRoutePathRewrite
    - HTTPRoutePortRedirect
    - HTTPRouteQueryParamMatching
    - HTTPRouteRequestMirror
    - HTTPRouteResponseHeaderModification
    - HTTPRouteSchemeRedirect
    unsupportedFeatures:
    - BackendTLSPolicy
    - BackendTLSPolicySANValidation
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - HTTPRoute303RedirectStatusCode
    - HTTPRoute307RedirectStatusCode
    - HTTPRoute308RedirectStatusCode
    - HTTPRouteBackendProtocolH2C
    - HTTPRouteBackendRequestHeaderModification
    - HTTPRouteBackendTimeout
    - HTTPRouteCORS
    - HTTPRouteNamedRouteRule
    - HTTPRouteParentRefPort
    - HTTPRoutePathRedirect
    - HTTPRouteRequestMultipleMirrors
    - HTTPRouteRequestPercentageMirror
    - HTTPRouteRequestTimeout
    - HTTPRouteRetry
    - HTTPRouteRetryBackendTimeout
    - HTTPRouteRetryConnectionError
    - ListenerSet
  name: GATEWAY-HTTP
  summary: Core tests partially succeeded with 7 test skips. Extended tests partially
    succeeded with 1 test skips.
- core:
    result: partial
    skippedTests:
    - GRPCRouteListenerHostnameMatching
    statistics:
      Failed: 0
      Passed: 14
      Skipped: 1
  extended:
    result: success
    statistics:
      Failed: 0
      Passed: 1
      Skipped: 0
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    unsupportedFeatures:
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - ListenerSet
  name: GATEWAY-GRPC
  summary: Core tests partially succeeded with 1 test skips. Extended tests succeeded.
- core:
    result: partial
    skippedTests:
    - TLSRouteHostnameIntersection
    - TLSRouteInvalidBackendRefNonexistent
    - TLSRouteInvalidBackendRefUnknownKind
    - TLSRouteSimpleSameNamespace
    statistics:
      Failed: 0
      Passed: 16
      Skipped: 4
  extended:
    result: partial
    skippedTests:
    - TLSRouteTerminateSimpleSameNamespace
    statistics:
      Failed: 0
      Passed: 3
      Skipped: 1
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    - TLSRouteModeTerminate
    unsupportedFeatures:
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - ListenerSet
    - TLSRouteModeMixed
  name: GATEWAY-TLS
  summary: Core tests partially succeeded with 4 test skips. Extended tests partially
    succeeded with 1 test skips.
succeededProvisionalTests:
- GatewayOptionalAddressValue

@github-actions

github-actions Bot commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

conformance test report

apiVersion: gateway.networking.k8s.io/v1
date: "2026-07-29T08:25:12Z"
gatewayAPIChannel: experimental
gatewayAPIVersion: v1.6.0
implementation:
  contact:
  - https://github.com/apache/apisix-ingress-controller/issues
  organization: APISIX
  project: apisix-ingress-controller
  url: https://github.com/apache/apisix-ingress-controller.git
  version: v2.0.0
kind: ConformanceReport
mode: default
profiles:
- core:
    failedTests:
    - GatewayModifyListeners
    result: failure
    statistics:
      Failed: 1
      Passed: 14
      Skipped: 0
  extended:
    result: success
    statistics:
      Failed: 0
      Passed: 1
      Skipped: 0
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    unsupportedFeatures:
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - ListenerSet
  name: GATEWAY-GRPC
  summary: Core tests failed with 1 test failures. Extended tests succeeded.
- core:
    failedTests:
    - GatewayModifyListeners
    - TLSRouteHostnameIntersection
    - TLSRouteInvalidBackendRefNonexistent
    - TLSRouteInvalidBackendRefUnknownKind
    - TLSRouteSimpleSameNamespace
    result: failure
    statistics:
      Failed: 5
      Passed: 15
      Skipped: 0
  extended:
    failedTests:
    - TLSRouteTerminateSimpleSameNamespace
    result: failure
    statistics:
      Failed: 1
      Passed: 3
      Skipped: 0
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    - TLSRouteModeTerminate
    unsupportedFeatures:
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - ListenerSet
    - TLSRouteModeMixed
  name: GATEWAY-TLS
  summary: Core tests failed with 5 test failures. Extended tests failed with 1 test
    failures.
- core:
    failedTests:
    - GatewayModifyListeners
    - HTTPRouteMultipleGateways
    - HTTPRouteNoBackendRefs
    result: failure
    skippedTests:
    - HTTPRouteHTTPSListener
    statistics:
      Failed: 3
      Passed: 33
      Skipped: 1
  extended:
    result: partial
    skippedTests:
    - HTTPRouteRedirectPortAndScheme
    statistics:
      Failed: 0
      Passed: 12
      Skipped: 1
    supportedFeatures:
    - GatewayAddressEmpty
    - GatewayPort8080
    - HTTPRouteBackendProtocolWebSocket
    - HTTPRouteDestinationPortMatching
    - HTTPRouteHostRewrite
    - HTTPRouteMethodMatching
    - HTTPRoutePathRewrite
    - HTTPRoutePortRedirect
    - HTTPRouteQueryParamMatching
    - HTTPRouteRequestMirror
    - HTTPRouteResponseHeaderModification
    - HTTPRouteSchemeRedirect
    unsupportedFeatures:
    - BackendTLSPolicy
    - BackendTLSPolicySANValidation
    - GatewayBackendClientCertificate
    - GatewayFrontendClientCertificateValidation
    - GatewayFrontendClientCertificateValidationInsecureFallback
    - GatewayHTTPListenerIsolation
    - GatewayHTTPSListenerDetectMisdirectedRequests
    - GatewayInfrastructurePropagation
    - GatewayStaticAddresses
    - HTTPRoute303RedirectStatusCode
    - HTTPRoute307RedirectStatusCode
    - HTTPRoute308RedirectStatusCode
    - HTTPRouteBackendProtocolH2C
    - HTTPRouteBackendRequestHeaderModification
    - HTTPRouteBackendTimeout
    - HTTPRouteCORS
    - HTTPRouteNamedRouteRule
    - HTTPRouteParentRefPort
    - HTTPRoutePathRedirect
    - HTTPRouteRequestMultipleMirrors
    - HTTPRouteRequestPercentageMirror
    - HTTPRouteRequestTimeout
    - HTTPRouteRetry
    - HTTPRouteRetryBackendTimeout
    - HTTPRouteRetryConnectionError
    - ListenerSet
  name: GATEWAY-HTTP
  summary: Core tests failed with 3 test failures. Extended tests partially succeeded
    with 1 test skips.
succeededProvisionalTests:
- GatewayOptionalAddressValue

@AlinsRan
feat: allow L4 upstream schemes (tcp/tls/udp) for stream backends
6e34fbf 
APISIX accepts `tcp`, `tls` and `udp` as upstream schemes for L4 proxying,
but both `ApisixUpstream.spec.scheme` and `BackendTrafficPolicy.spec.scheme`
restricted the value to the L7 set, so an upstream that requires TLS from
APISIX (plain TCP in, TLS out) could only be configured through the Admin API.

The translation path already carries the scheme to the generated upstream for
both ApisixRoute stream backends and Gateway API TCPRoute backends, so only
the CRD validation needed to be relaxed.

Sync of apache/apisix-ingress-controller#2830 (fixes apache/apisix-ingress-controller#2803).
@AlinsRan
AlinsRan force-pushed the ee-2803-l4-upstream-scheme branch from 075d13c to 6e34fbf Compare July 29, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Question: ApisixRoute CRD stream backend lacks scheme: tls — is Admin API safe to use alongside CRDs?

1 participant

@AlinsRan