docs: surface Redpanda Connect pipelines for operator users (DOC-2275)#1808
Open
JakeSCahill wants to merge 8 commits into
Open
docs: surface Redpanda Connect pipelines for operator users (DOC-2275)#1808JakeSCahill wants to merge 8 commits into
JakeSCahill wants to merge 8 commits into
Conversation
Three discovery surfaces for the 26.2 Pipeline CRD (DOC-2275): - new Manage > Kubernetes page routing to the full guide in the Connect docs, positioned next to Manage Kafka Connect - Pipeline row in the CRD reference index - TIP on Manage Kafka Connect pointing at the first-class alternative
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Author
|
Update: the Redpanda Connect operator support (Pipeline CRD, redpanda-operator#1677) is expected to slip past 26.2, most likely to 26.3. This PR stays in draft until the CRD ships in a released operator. Do not merge for the 26.2 GA on July 28. |
Contributor
|
PR is merged redpanda-data/redpanda-operator#1677, this will go out as Beta for the 26.2 date. Could you ensure that examples are provided with examples of how to use HPA and KEDA for autoscaling? |
…censing, and what's new (DOC-2275) Turn the discovery page into a full operator-side guide covering every flow from redpanda-operator#1677: enable the Connect controller, first pipeline, clusterRef+userRef cluster binding (with the rendered-config injection contract), staticConfiguration for external clusters, per-pipeline cloud IAM via serviceAccountName (IRSA worked example), and valueSources with all four source types. Adds status/condition reference and troubleshooting. Add a dedicated autoscaling page with step-by-step HPA (cpu/memory) and KEDA (consumer-group lag, Connect metrics via Prometheus, scale-to-zero) examples, all grounded in manifests validated end-to-end on EKS. Document the no-license/expired-license behavior for pipelines in the licensing overview, and add the operator what's-new entry (version section marked TODO until the ship vehicle is confirmed). Route the CRD index row to the local guide instead of the unpublished connect-docs page, which removes the two expected broken-xref warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
david-yu
marked this pull request as ready for review
July 24, 2026 18:09
The feature ships with operator v26.2.x, not v26.3.x. Drop the provisional v26.3.x section and its TODO. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ing overview Give the operator its own enterprise-features table, matching the Redpanda / Console / Connect sections: Stretch Clusters (multicluster operator refuses to start without a valid license; LicenseValid condition + status.licenseStatus on running clusters) and Connect pipelines (moved out of the Connect table). Add a 'Manage licenses for the Redpanda Operator' section covering enterprise.licenseSecretRef and per-cluster licensing via the Redpanda and StretchCluster resources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ect product entitlement Replace the bespoke license sentence on the pipelines page with the shared enterprise-license partial, and state everywhere the license is discussed (notice, prerequisites, license behavior, troubleshooting, licensing overview table) that the license must include the Redpanda Connect product — matching the operator's IncludesProduct(ProductConnect) check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
david-yu
approved these changes
Jul 24, 2026
Add an 'Expose Redpanda Console with the Gateway API' section to the Console Kubernetes deploy page (operator PR #1329): Gateway API CRD + controller + Gateway prerequisites, Operator CRD and Helm tabs for the gateway block, route verification (Accepted/ResolvedRefs, allowedRoutes gotcha), and switching between Ingress and Gateway API. Both Ingress callouts now point at the new section, and the operator v26.2.x what's-new gains an entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DRAFT: hold until the Pipeline CRD ships in a released operator. Ships with operator v26.2.x.
Documents the Pipeline resource (redpanda-operator#1677) for operator users. Originally a discovery-only PR; now expanded into the full operator-side guide, per review feedback, covering every flow in the eng PR description with worked, e2e-validated examples.
Pages
manage:kubernetes/k-connect-pipelines.adoc(nav: "Redpanda Connect Pipelines") — full guide:connectController.enabled+enterprise.licenseSecretRef)configYaml), lint-gated startup, config/credential checksum rollouts, pausecluster.clusterRef+userRefwith a scopedUserCR, the rendered-config injection contract (seed_brokers/tls/sasl+ top-level shared client), user-keys-win escape hatch, ACL-scoping guidancecluster.staticConfigurationwith Secret/ConfigMap-backed TLS CA, inline SASL, CEL exclusivity withuserRefserviceAccountName+ IRSA worked example (RDS IAM auth, no DB password anywhere)inline/secretKeyRef/configMapKeyRef/externalSecretRef) in one fan-out exampleextraInitContainers), monitoring (PodMonitor), license behavior, status/conditions reference, troubleshooting. Uses the standardshared:partial$enterprise-license.adocnotice and states that the license must include the Redpanda Connect product (matches the operator'sIncludesProduct(ProductConnect)check).manage:kubernetes/k-autoscale-connect-pipelines.adoc(new, nav: nested "Autoscaling") — step-by-step autoscaling through the scale subresource, grounded in manifests validated e2e on EKS (HPA cpu 1→4; KEDA kafka-lag 1→6; KEDA prometheusinput_received1→3):kubectl scale+ raw/scalesmoke testkubectl get hpaoutput)connectController.monitoring.enabled+ PodMonitor), scale-to-zeropausedprecedence, explicitreplicas: 0, GitOps ownership ofreplicas, one autoscaler per pipelineget-started:licensing/overview.adoc): new "Redpanda Operator enterprise features" section with its own table (matching the Redpanda/Console/Connect sections) covering both operator-gated features — Stretch Clusters (multicluster operator requires a valid license to start;LicenseValidcondition +status.licenseStatusreport expiry/violations on running clusters) and Connect pipelines (missing/expired license or missing Connect product entitlement → new pipelines reportLicense: Falseand get no workload; running pipelines keep processing but stop receiving updates; recovery is automatic once a valid license is applied). Also adds a "Manage licenses for the Redpanda Operator" section (enterprise.licenseSecretRefchart values + per-cluster licensing via the Redpanda/StretchClusterenterprisefield).get-started:release-notes/operator.adoc): "Redpanda Connect pipelines (beta)" and "Gateway API support for Redpanda Console" entries under thev26.2.xsection.deploy:console/kubernetes/deploy.adoc, from redpanda-operator#1329): new "Expose Redpanda Console with the Gateway API" section — Gateway API CRD/controller/Gateway prerequisites, Operator CRD + Helmgatewayblock examples, route verification (Accepted/ResolvedRefs,allowedRoutesgotcha), Ingress↔Gateway switching, and mutual-exclusivity note; both Ingress callouts route to it.Build note
The two previously-expected broken-xref warnings for
connect:install:kubernetes-operator.adocare gone: all routing now targets the local pages.origin/betahas been merged into the branch.Before un-drafting (GA batch)
k-connect-pipelines.adoc(TODO marker; expected v26.2.1)connect:install:kubernetes-operator.adoc(comment markers in place)Part of DOC-2275; eng feature is redpanda-operator#1677.
Preview pages
🤖 Generated with Claude Code