Skip to content

feat(ticdc): prevent using the same TiDB cluster as both upstream and downstream#3956

Merged
ti-chi-bot[bot] merged 9 commits intopingcap:masterfrom
wlwilliamx:feat/prevent-same-upstream-downstream
Jan 25, 2026
Merged

feat(ticdc): prevent using the same TiDB cluster as both upstream and downstream#3956
ti-chi-bot[bot] merged 9 commits intopingcap:masterfrom
wlwilliamx:feat/prevent-same-upstream-downstream

Conversation

@wlwilliamx
Copy link
Copy Markdown
Collaborator

@wlwilliamx wlwilliamx commented Jan 8, 2026

What problem does this PR solve?

TiCDC does not support using the same TiDB logical cluster as both the upstream and downstream, because it can easily lead to self-replication loops or unexpected data behaviors.

In TiDB Next-Gen, a physical cluster can contain multiple logical clusters (keyspaces). Different keyspaces should be treated as different clusters from TiCDC's perspective. So this PR keeps rejecting same upstream/downstream for the same keyspace, while allowing cross-keyspace replication within the same physical cluster.

Issue Number: close #3949

What is changed and how it works?

  • Add a pre-check for changefeed create/update/resume to detect “same upstream & downstream” for MySQL-compatible sinks.
    • The downstream TiDB cluster ID is read from mysql.tidb (cluster_id).
    • If downstream is not TiDB or the cluster ID is unavailable, the check is skipped (legacy/compat behavior).
  • Next-Gen compatibility: when the upstream and downstream cluster_id are the same, query the downstream keyspace via:
    • show config where type = 'tidb' and name = 'keyspace-name'
    • Treat (cluster_id, keyspace) as the cluster identity, so cross-keyspace is allowed but same keyspace is rejected.
    • If downstream keyspace cannot be determined, keep the conservative legacy behavior (treat as the same physical cluster and reject) to avoid accidental self-replication.
  • Add unit tests and an integration test case same_upstream_downstream.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test

Questions

Will it cause performance regression or break compatibility?
  • No steady-state overhead: the check only runs when creating/updating/resuming a changefeed.
  • Classic mode keeps the existing behavior (same physical cluster is rejected).
  • Next-Gen allows cross-keyspace replication within the same physical cluster, while still rejecting same keyspace.
Do you need to update user documentation, design documentation or monitoring documentation?

N/A

Release note

 Reject creating/updating/resuming a changefeed when downstream is the same TiDB logical cluster as upstream (keyspace-aware for TiDB Next-Gen).

@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/needs-triage-completed labels Jan 8, 2026
@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello @wlwilliamx, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances TiCDC's robustness by implementing a critical safeguard against self-replication. It ensures that a TiDB cluster cannot serve as both the data source and the replication target for a changefeed, thereby preventing potential data inconsistencies and operational issues. The change integrates a new cluster identification and comparison logic into the changefeed lifecycle management, providing immediate feedback and preventing misconfigurations.

Highlights

  • Preventing Self-Replication: Introduced a new validation mechanism to prevent a TiDB cluster from being configured as both the upstream source and downstream target for a TiCDC changefeed.
  • Validation Scope: The upstream/downstream cluster identity check is now enforced during changefeed creation, update, and resume operations to ensure data integrity and prevent circular replication.
  • New Cluster Check Utility: A new utility function UpstreamDownstreamNotSame has been added to pkg/check to determine if the upstream and downstream TiDB clusters are distinct by comparing their cluster IDs.
  • Error Handling: A specific error, ErrSameUpstreamDownstream, is now returned when this self-replication condition is detected.
  • Comprehensive Testing: New unit tests for the cluster ID checking logic and an integration test script have been added to validate the new behavior across create, update, and resume scenarios.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@wlwilliamx wlwilliamx marked this pull request as draft January 8, 2026 06:26
@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Jan 8, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a valuable feature to prevent a changefeed from using the same TiDB cluster as both upstream and downstream, which could lead to replication loops. The implementation correctly adds checks at changefeed creation, update, and resume. The core logic for comparing cluster IDs is well-encapsulated in the new pkg/check package and is accompanied by good unit and integration tests.

My main feedback is regarding code duplication in api/v2/changefeed.go, where the check logic is repeated in three different API handlers. Refactoring this into a shared helper function would improve maintainability. I've also included a couple of minor suggestions for improving code clarity and error handling in pkg/check/cluster.go.

Overall, this is a solid contribution that enhances the robustness of TiCDC.

Comment thread api/v2/changefeed.go Outdated
Comment thread pkg/check/cluster.go
Comment thread pkg/check/cluster.go Outdated
@ti-chi-bot ti-chi-bot Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. do-not-merge/needs-linked-issue do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/needs-triage-completed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jan 14, 2026
@wlwilliamx wlwilliamx marked this pull request as ready for review January 14, 2026 11:47
@ti-chi-bot ti-chi-bot Bot removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/needs-linked-issue labels Jan 14, 2026
@wlwilliamx
Copy link
Copy Markdown
Collaborator Author

CC @tenfyzhong

@wlwilliamx
Copy link
Copy Markdown
Collaborator Author

CC @asddongmen

Comment thread api/v2/changefeed.go Outdated
Comment thread pkg/check/cluster.go
Comment thread pkg/check/cluster.go
Comment thread pkg/check/cluster.go Outdated
@ti-chi-bot ti-chi-bot Bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Jan 21, 2026
@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jan 23, 2026
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Jan 23, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tenfyzhong, wk989898

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [tenfyzhong,wk989898]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot Bot commented Jan 23, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-01-21 14:51:08.876568421 +0000 UTC m=+591896.490525277: ☑️ agreed by tenfyzhong.
  • 2026-01-23 09:11:53.60733012 +0000 UTC m=+744341.221286975: ☑️ agreed by wk989898.

@wlwilliamx
Copy link
Copy Markdown
Collaborator Author

/test all

@ti-chi-bot ti-chi-bot Bot merged commit e081f53 into pingcap:master Jan 25, 2026
24 checks passed
lidezhu pushed a commit that referenced this pull request Jan 31, 2026
a-cong pushed a commit to a-cong/ticdc that referenced this pull request Feb 2, 2026
tenfyzhong pushed a commit that referenced this pull request Mar 18, 2026
… downstream (#3956)

close #3949

(cherry picked from commit e081f53)
Signed-off-by: tenfyzhong <tenfy@tenfy.cn>
tenfyzhong pushed a commit that referenced this pull request Mar 18, 2026
tenfyzhong pushed a commit that referenced this pull request Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent TiCDC from Using the Same TiDB Cluster as Both Upstream and Downstream

3 participants