docs: add supported connection paths for remote clusters (CCS/CCR vs private connectivity) - #7567
docs: add supported connection paths for remote clusters (CCS/CCR vs private connectivity)#7567alxchalkias wants to merge 2 commits into
Conversation
|
💚 CLA has been signed |
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
🔍 Preview links for changed docs |
eedugon
left a comment
There was a problem hiding this comment.
Great addition to the docs, thanks!!
I've shared a few comments, suggesting a small update in the title, because the main focus on our messages are about private connectivity.
There's one sentence that was contradictory with the rest of the PR and I've suggested to change the introductory path of the section (including a link to private connectivity docs, as we didn't have any in the section).
Let me know your thoughts!
Note that I've removed in the intro the message about 9400, 9443, API key, and TLS-based auth methods, as IMO that's out of the scope of this doc and it's explained already in all the setup docs plus the doc about authentication methods for RCS. But if we consider it's useful to add it also here we can find the way.
jakommo
left a comment
There was a problem hiding this comment.
Left a few minor / clarification comment, but LGTM in general.
| :::: | ||
|
|
||
| ::::{tip} | ||
| Traffic sent between deployments for {{ccs}} and {{ccr}} is metered as **Data out** on the deployment the data is replicated or searched from, regardless of destination or path. Refer to [Cloud Hosted deployment billing dimensions](/deploy-manage/cloud-organization/billing/cloud-hosted-deployment-billing-dimensions.md). |
There was a problem hiding this comment.
or searched from
I think this is ambiguous and might cause confusion. For CCS is this the cluster initiating the search or the one that executes it?
There was a problem hiding this comment.
Good catch — reworded. It's the remote deployment being searched (the search results leave that deployment), not the initiating cluster. The tip now spells out both directions: for CCR the deployment you replicate from, for CCS the remote deployment being searched.
| | {{ech}} deployment | Self-managed cluster in your network | Outbound from Elastic over the public internet to your cluster's endpoint. Allow the connection with IP-based rules on your side. | **No.** {{ech}} deployments can't consume private endpoints in customer networks. | | ||
|
|
||
| ::::{warning} | ||
| Don't use a private connection hostname (for example, `*.vpce.{region}.aws.elastic-cloud.com` or your Azure private hosted zone domain) as the `proxy_address` for a remote cluster connection between two {{ech}} deployments. The connection will fail because the hostname is not resolvable from Elastic-managed networks. Use the public proxy address from the remote deployment's **Security** page instead. |
There was a problem hiding this comment.
Is there a specific error for this? Something like "can't resolve" or "host unknown" or so? Would be great to add the symptom/error so users can more easily find the doc.
There was a problem hiding this comment.
Added. The symptom is a DNS resolution failure — unknown host / UnknownHostException in the Elasticsearch logs — since the private hostname isn't resolvable from Elastic-managed networks. The warning now includes it for searchability.
|
I think this PR will always fail CI because of the cursoragent commit ... might need to be recreated |
f18d8e1 to
395be1e
Compare
|
Addressed all open review items and rebuilt the branch per @shainaraskas' CLA note:
CI should be green now; ready for another look. |
Add a Connection paths and private connectivity subsection to the remote clusters network security section, clarifying which paths support CCS/CCR by local cluster location and that private connection hostnames cannot be used between two ECH deployments. Add a corresponding limitation note to the shared private connectivity limitations snippet (AWS/Azure/GCP), and a billing tip pointing at the Data out dimension. Co-authored-by: Edu González de la Herrán <25320357+eedugon@users.noreply.github.com> Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
395be1e to
628e132
Compare
nikfot
left a comment
There was a problem hiding this comment.
Added some comments for clarity.
| @@ -1,4 +1,5 @@ | |||
| * **Transport client:** The {{es}} transport client is not supported over private connections. | |||
| * **Remote clusters between {{ech}} deployments:** Remote cluster connections ({{ccs}} and {{ccr}}) between two ECH deployments cannot use your private endpoint. Deployment-to-deployment traffic must use the remote deployment's public proxy address. Private connections apply to remote cluster traffic only when the local cluster runs in your own VPC or VNet (self-managed, {{ece}}, or {{eck}}). Refer to [Supported connection paths](/deploy-manage/remote-clusters.md#remote-clusters-connection-paths). | |||
There was a problem hiding this comment.
I believe that "your private endpoint" can be misleading. Since this is in Elastic Cloud the user cannot create a private endpoint no matter what.
Maybe something like this:
| * **Remote clusters between {{ech}} deployments:** Remote cluster connections ({{ccs}} and {{ccr}}) between two ECH deployments cannot use your private endpoint. Deployment-to-deployment traffic must use the remote deployment's public proxy address. Private connections apply to remote cluster traffic only when the local cluster runs in your own VPC or VNet (self-managed, {{ece}}, or {{eck}}). Refer to [Supported connection paths](/deploy-manage/remote-clusters.md#remote-clusters-connection-paths). | |
| * **Remote clusters between {{ech}} deployments:** Remote cluster connections ({{ccs}} and {{ccr}}) between two ECH deployments cannot be connecte using private endpoints. Deployment-to-deployment traffic must use the remote deployment's public proxy address. Private connections apply to remote cluster traffic only when the local cluster runs in your own VPC or VNet (self-managed, {{ece}}, or {{eck}}). Refer to [Supported connection paths](/deploy-manage/remote-clusters.md#remote-clusters-connection-paths). |
|
|
||
| | Local cluster location | Remote cluster | Connection path | Private connectivity applicable? | | ||
| |---|---|---|---| | ||
| | {{ech}} deployment (any region or cloud provider) | {{ech}} deployment | Remote proxy address over public endpoints. Cross-region and cross-provider connections are supported. | **No.** Private connection hostnames resolve only inside your VPC or VNet through your private DNS zone. Deployment-to-deployment traffic originates from Elastic-managed networks and can't traverse your private endpoint. | |
There was a problem hiding this comment.
The problem is not that the hostnames would not resolve. Ther problem is that the user does not have permission to create them inside the Elastic CLoud.
| | {{ech}} deployment (any region or cloud provider) | {{ech}} deployment | Remote proxy address over public endpoints. Cross-region and cross-provider connections are supported. | **No.** Private connection hostnames resolve only inside your VPC or VNet through your private DNS zone. Deployment-to-deployment traffic originates from Elastic-managed networks and can't traverse your private endpoint. | | |
| | {{ech}} deployment (any region or cloud provider) | {{ech}} deployment | Remote proxy address over public endpoints. Cross-region and cross-provider connections are supported. | **No.** You can only create private connection hostnames inside your VPC or VNet through your private DNS zone. Deployment-to-deployment traffic originates from Elastic-managed networks and can't traverse your private endpoint. | |
| | Local cluster location | Remote cluster | Connection path | Private connectivity applicable? | | ||
| |---|---|---|---| | ||
| | {{ech}} deployment (any region or cloud provider) | {{ech}} deployment | Remote proxy address over public endpoints. Cross-region and cross-provider connections are supported. | **No.** Private connection hostnames resolve only inside your VPC or VNet through your private DNS zone. Deployment-to-deployment traffic originates from Elastic-managed networks and can't traverse your private endpoint. | | ||
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC endpoint (AWS), private endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (in this direction only). | |
There was a problem hiding this comment.
To keep capitalization common across services
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC endpoint (AWS), private endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (in this direction only). | | |
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC Endpoint (AWS), Private Endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (in this direction only). | |
| |---|---|---|---| | ||
| | {{ech}} deployment (any region or cloud provider) | {{ech}} deployment | Remote proxy address over public endpoints. Cross-region and cross-provider connections are supported. | **No.** Private connection hostnames resolve only inside your VPC or VNet through your private DNS zone. Deployment-to-deployment traffic originates from Elastic-managed networks and can't traverse your private endpoint. | | ||
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC endpoint (AWS), private endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (in this direction only). | | ||
| | {{ech}} deployment | Self-managed cluster in your network | Outbound from Elastic over the public internet to your cluster's endpoint. Allow the connection with IP-based rules on your side. | **No.** {{ech}} deployments can't consume private endpoints in customer networks. | |
There was a problem hiding this comment.
| | {{ech}} deployment | Self-managed cluster in your network | Outbound from Elastic over the public internet to your cluster's endpoint. Allow the connection with IP-based rules on your side. | **No.** {{ech}} deployments can't consume private endpoints in customer networks. | | |
| | {{ech}} deployment | Self-managed cluster in your network | Outbound from Elastic over the public internet to your cluster's endpoint. Allow the connection with IP-based rules on your side. | **No.** {{ech}} deployments can't create private endpoints for customer networks. | |
| | Local cluster location | Remote cluster | Connection path | Private connectivity applicable? | | ||
| |---|---|---|---| | ||
| | {{ech}} deployment (any region or cloud provider) | {{ech}} deployment | Remote proxy address over public endpoints. Cross-region and cross-provider connections are supported. | **No.** Private connection hostnames resolve only inside your VPC or VNet through your private DNS zone. Deployment-to-deployment traffic originates from Elastic-managed networks and can't traverse your private endpoint. | | ||
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC endpoint (AWS), private endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (in this direction only). | |
There was a problem hiding this comment.
I don't think that "in this direction only" is very clear. What direction?
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC endpoint (AWS), private endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (in this direction only). | | |
| | Self-managed or {{ece}} cluster in your VPC or VNet | {{ech}} deployment | Public proxy address, or a private connection: create a VPC endpoint (AWS), private endpoint (Azure), or Private Service Connect endpoint (GCP) and connect to the remote cluster through it. Inter-region private connections are supported where the cloud provider supports them. | **Yes** (customer network → ECH only). | |
Summary
deploy-manage/remote-clusters.md, with a table showing which network paths support CCS/CCR depending on where the local cluster runs, a warning against using private connection hostnames asproxy_addressbetween two ECH deployments, and a billing tip (CCS/CCR traffic is metered as Data out on the deployment the data leaves)._snippets/private-connectivity-limitations-ech.md), which renders on the AWS PrivateLink, Azure Private Link, and GCP Private Service Connect pages: deployment-to-deployment remote cluster traffic does not traverse the customer's private endpoint.Why
Recent support escalation (SDH via #cloud-networking Slack, Jul 17-24): a customer tried to configure CCR between two ECH deployments in Azure (
eastus<->westus2) using the Private Link hostname asproxy_address, which cannot work — PL hostnames only resolve inside the customer's VNet. The existing docs cover which filter type to use when network security is enabled, but not which connection paths are supported, and both support and the customer were confused about::9400is open on the Azure PL LBs — ECPTRAFFIC-1076 — and:9443/RCS 2.0 was addressed in ECPTRAFFIC-1293).Test plan
deploy-manage/remote-clusters#remote-clusters-connection-pathsresolves from the snippet linkMade with Cursor