From c88627ff5ab6cc194dd06c5d492a63e53ed04416 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Sun, 8 Mar 2026 09:45:59 +0100 Subject: [PATCH 1/5] Update for DM related docs --- dm/dm-compatibility-catalog.md | 24 +++++++------- dm/dm-precheck.md | 33 ++++++++++--------- ...migrate-from-mysql-using-data-migration.md | 24 +++++++------- 3 files changed, 42 insertions(+), 39 deletions(-) diff --git a/dm/dm-compatibility-catalog.md b/dm/dm-compatibility-catalog.md index c7a28d31ed256..08f6ec77d3bcc 100644 --- a/dm/dm-compatibility-catalog.md +++ b/dm/dm-compatibility-catalog.md @@ -14,18 +14,18 @@ DM supports migrating data from different sources to TiDB clusters. Based on the ## Data sources -| Data source | Compatibility level | Note | -| - | - | - | -| MySQL ≤ 5.5 | Not tested | | -| MySQL 5.6 | GA | | -| MySQL 5.7 | GA | | -| MySQL 8.0 | GA | Does not support binlog transaction compression [Transaction_payload_event](https://dev.mysql.com/doc/refman/8.0/en/binary-log-transaction-compression.html). | -| MySQL 8.1 ~ 8.3 | Not tested | | -| MySQL 8.4 | Incompatible | For more information, see [DM Issue #11020](https://github.com/pingcap/tiflow/issues/11020). | -| MySQL 9.x | Not tested | | -| MariaDB < 10.1.2 | Incompatible | Incompatible with binlog of the time type. | -| MariaDB 10.1.2 ~ 10.5.10 | Experimental | | -| MariaDB > 10.5.10 | Not tested | Expected to work in most cases after bypassing the [precheck](/dm/dm-precheck.md). See [MariaDB notes](#mariadb-notes). | +| Data source | Compatibility level | Note | +| ------------------------ | ------------------- | ---- | +| MySQL ≤ 5.5 | Not tested | | +| MySQL 5.6 | GA | | +| MySQL 5.7 | GA | | +| MySQL 8.0 | GA | Does not support binlog transaction compression [Transaction_payload_event](https://dev.mysql.com/doc/refman/8.0/en/binary-log-transaction-compression.html). | +| MySQL 8.1 ~ 8.3 | Not tested | | +| MySQL 8.4 | Incompatible | For more information, see [DM Issue #11020](https://github.com/pingcap/tiflow/issues/11020). | +| MySQL 9.x | Not tested | | +| MariaDB < 10.1.2 | Incompatible | Incompatible with binlog of the time type. | +| MariaDB 10.1.2 ~ 10.5.10 | Experimental | | +| MariaDB > 10.5.10 | Not tested | Expected to work in most cases after bypassing the [precheck](/dm/dm-precheck.md). See [MariaDB notes](#mariadb-notes). | ### Incompatibility with foreign key CASCADE operations diff --git a/dm/dm-precheck.md b/dm/dm-precheck.md index 48b41f16a6e09..71880c6c67747 100644 --- a/dm/dm-precheck.md +++ b/dm/dm-precheck.md @@ -52,7 +52,7 @@ Regardless of the migration mode you choose, the precheck always includes the fo - Compatibility of the upstream MySQL table schema - - Check whether the upstream tables have foreign keys, which are not supported by TiDB. A warning is returned if a foreign key is found in the precheck. + - Check whether the upstream tables have foreign keys, which are not fully supported by TiDB. A warning is returned if a foreign key is found in the precheck. - Check whether the upstream tables use character sets that are incompatible with TiDB. For more information, see [TiDB Supported Character Sets](/character-set-and-collation.md). - Check whether the upstream tables have primary key constraints or unique key constraints (introduced from v1.0.7). @@ -132,6 +132,7 @@ For the incremental data migration mode (`task-mode: incremental`), in addition - Check whether binlog is enabled (required by DM). - Check whether `binlog_format=ROW` is configured (DM only supports the migration of binlog in the ROW format). - Check whether `binlog_row_image=FULL` is configured (DM only supports `binlog_row_image=FULL`). + - Check whether `binlog_transaction_compression=OFF` is configured (DM doesn't support transaction compression). - If `binlog_do_db` or `binlog_ignore_db` is configured, check whether the database tables to be migrated meet the conditions of `binlog_do_db` and `binlog_ignore_db`. * (Mandatory) Check if the upstream database is in an [Online-DDL](/dm/feature-online-ddl.md) process (in which the `ghost` table is created but the `rename` phase is not executed yet). If the upstream is in the online-DDL process, the precheck returns an error. In this case, wait until the DDL to complete and retry. @@ -144,21 +145,21 @@ For the full and incremental data migration mode (`task-mode: all`), in addition Prechecks can find potential risks in your environments. It is not recommended to ignore check items. If your data migration task has special needs, you can use the [`ignore-checking-items` configuration item](/dm/task-configuration-file-full.md#task-configuration-file-template-advanced) to skip some check items. -| Check item | Description | -| :---------- | :------------ | -| `dump_privilege` | Checks the dump privilege of the user in the upstream MySQL instance. | -| `replication_privilege` | Checks the replication privilege of the user in the upstream MySQL instance. | -| `version` | Checks the version of the upstream database. | -| `server_id` | Checks whether server_id is configured in the upstream database. | -| `binlog_enable` | Checks whether binlog is enabled in the upstream database. | -| `table_schema` | Checks the compatibility of the table schemas in the upstream MySQL tables. | -| `schema_of_shard_tables`| Checks the consistency of the table schemas in the upstream MySQL multi-instance shards. | -| `auto_increment_ID` | Checks whether the auto-increment primary key conflicts in the upstream MySQL multi-instance shards. | -|`online_ddl`| Checks whether the upstream is in the process of [online-DDL](/dm/feature-online-ddl.md). | -| `empty_region` | Checks the number of empty Regions in the downstream database for physical import. | -| `region_distribution` | Checks the distribution of Regions in the downstream database for physical import. | -| `downstream_version` | Checks the versions of TiDB, PD, and TiKV in the downstream database. | -| `free_space` | Checks the free space of the downstream database. | +| Check item | Description | +| :-------------------------- | :------------ | +| `dump_privilege` | Checks the dump privilege of the user in the upstream MySQL instance. | +| `replication_privilege` | Checks the replication privilege of the user in the upstream MySQL instance. | +| `version` | Checks the version of the upstream database. | +| `server_id` | Checks whether server_id is configured in the upstream database. | +| `binlog_enable` | Checks whether binlog is enabled in the upstream database. | +| `table_schema` | Checks the compatibility of the table schemas in the upstream MySQL tables. | +| `schema_of_shard_tables` | Checks the consistency of the table schemas in the upstream MySQL multi-instance shards. | +| `auto_increment_ID` | Checks whether the auto-increment primary key conflicts in the upstream MySQL multi-instance shards. | +| `online_ddl` | Checks whether the upstream is in the process of [online-DDL](/dm/feature-online-ddl.md). | +| `empty_region` | Checks the number of empty Regions in the downstream database for physical import. | +| `region_distribution` | Checks the distribution of Regions in the downstream database for physical import. | +| `downstream_version` | Checks the versions of TiDB, PD, and TiKV in the downstream database. | +| `free_space` | Checks the free space of the downstream database. | | `downstream_mutex_features` | Checks whether the downstream database is running tasks that are incompatible with physical import. | > **Note:** diff --git a/tidb-cloud/migrate-from-mysql-using-data-migration.md b/tidb-cloud/migrate-from-mysql-using-data-migration.md index d708402cb6502..a62fee25125e4 100644 --- a/tidb-cloud/migrate-from-mysql-using-data-migration.md +++ b/tidb-cloud/migrate-from-mysql-using-data-migration.md @@ -127,12 +127,12 @@ For {{{ .essential }}}, the Data Migration feature supports the following data s | Data source | Supported versions | |:-------------------------------------------------|:-------------------| -| Self-managed MySQL (on-premises or public cloud) | 8.0, 5.7 | -| Amazon Aurora MySQL | 8.0, 5.7 | +| Self-managed MySQL (on-premises or public cloud) | 8.0, 5.7 | +| Amazon Aurora MySQL | 8.0, 5.7 | | Amazon RDS MySQL | 8.0, 5.7 | | Alibaba Cloud RDS MySQL | 8.0, 5.7 | -| Azure Database for MySQL - Flexible Server | 8.0, 5.7 | -| Google Cloud SQL for MySQL | 8.0, 5.7 | +| Azure Database for MySQL - Flexible Server | 8.0, 5.7 | +| Google Cloud SQL for MySQL | 8.0, 5.7 | @@ -140,12 +140,13 @@ For {{{ .essential }}}, the Data Migration feature supports the following data s To continuously replicate incremental changes from the source MySQL-compatible database to the TiDB Cloud target cluster using DM, you need the following configurations to enable binary logs in the source database: -| Configuration | Required value | Why | -|:--------------|:---------------|:----| -| `log_bin` | `ON` | Enables binary logging, which DM uses to replicate changes to TiDB | -| `binlog_format` | `ROW` | Captures all data changes accurately (other formats miss edge cases) | -| `binlog_row_image` | `FULL` | Includes all column values in events for safe conflict resolution | -| `binlog_expire_logs_seconds` | ≥ `86400` (1 day), `604800` (7 days, recommended) | Ensures DM can access consecutive logs during migration | +| Configuration | Required value | Why | +|:---------------------------------|:---------------|:----| +| `log_bin` | `ON` | Enables binary logging, which DM uses to replicate changes to TiDB | +| `binlog_format` | `ROW` | Captures all data changes accurately (other formats miss edge cases) | +| `binlog_row_image` | `FULL` | Includes all column values in events for safe conflict resolution | +| `binlog_expire_logs_seconds` | ≥ `86400` (1 day), `604800` (7 days, recommended) | Ensures DM can access consecutive logs during migration | +| `binlog_transaction_compression` | `OFF` | Transaction compression is not supported by DM | #### Check current values and configure the source MySQL instance @@ -154,7 +155,7 @@ To check the current configurations, connect to the source MySQL instance and ex ```sql SHOW VARIABLES WHERE Variable_name IN ('log_bin','server_id','binlog_format','binlog_row_image', -'binlog_expire_logs_seconds','expire_logs_days'); +'binlog_expire_logs_seconds','expire_logs_days','binlog_transaction_compression'); ``` If necessary, change the source MySQL instance configurations to match the required values. @@ -170,6 +171,7 @@ If necessary, change the source MySQL instance configurations to match the requi binlog_format = ROW binlog_row_image = FULL binlog_expire_logs_seconds = 604800 # 7 days retention + binlog_transaction_compression = OFF ``` 2. Restart the MySQL service to apply the changes: From 3e745b73f8d8acf2b91c134e72ca68c004186e4e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Sun, 8 Mar 2026 10:43:08 +0100 Subject: [PATCH 2/5] Update dm/dm-precheck.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- dm/dm-precheck.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dm/dm-precheck.md b/dm/dm-precheck.md index 71880c6c67747..280a7cacecfab 100644 --- a/dm/dm-precheck.md +++ b/dm/dm-precheck.md @@ -132,7 +132,7 @@ For the incremental data migration mode (`task-mode: incremental`), in addition - Check whether binlog is enabled (required by DM). - Check whether `binlog_format=ROW` is configured (DM only supports the migration of binlog in the ROW format). - Check whether `binlog_row_image=FULL` is configured (DM only supports `binlog_row_image=FULL`). - - Check whether `binlog_transaction_compression=OFF` is configured (DM doesn't support transaction compression). + - Check whether `binlog_transaction_compression=OFF` is configured (DM does not support transaction compression). - If `binlog_do_db` or `binlog_ignore_db` is configured, check whether the database tables to be migrated meet the conditions of `binlog_do_db` and `binlog_ignore_db`. * (Mandatory) Check if the upstream database is in an [Online-DDL](/dm/feature-online-ddl.md) process (in which the `ghost` table is created but the `rename` phase is not executed yet). If the upstream is in the online-DDL process, the precheck returns an error. In this case, wait until the DDL to complete and retry. From 7d4a4efb07a9e4ceefe076aaa32c770b68a312a2 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 9 Mar 2026 11:06:51 +0800 Subject: [PATCH 3/5] Update tidb-cloud/migrate-from-mysql-using-data-migration.md --- tidb-cloud/migrate-from-mysql-using-data-migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-cloud/migrate-from-mysql-using-data-migration.md b/tidb-cloud/migrate-from-mysql-using-data-migration.md index a62fee25125e4..0a6be421edf23 100644 --- a/tidb-cloud/migrate-from-mysql-using-data-migration.md +++ b/tidb-cloud/migrate-from-mysql-using-data-migration.md @@ -146,7 +146,7 @@ To continuously replicate incremental changes from the source MySQL-compatible d | `binlog_format` | `ROW` | Captures all data changes accurately (other formats miss edge cases) | | `binlog_row_image` | `FULL` | Includes all column values in events for safe conflict resolution | | `binlog_expire_logs_seconds` | ≥ `86400` (1 day), `604800` (7 days, recommended) | Ensures DM can access consecutive logs during migration | -| `binlog_transaction_compression` | `OFF` | Transaction compression is not supported by DM | +| `binlog_transaction_compression` | `OFF` | DM does not support transaction compression | #### Check current values and configure the source MySQL instance From 37f32ab00b20fd33f52821d1159a10b48e9ab44d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Mon, 9 Mar 2026 07:44:07 +0100 Subject: [PATCH 4/5] Try to fix linkChecker errors --- .lycheeignore | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.lycheeignore b/.lycheeignore index 97a8b48e871d5..fdcc00794a2e3 100644 --- a/.lycheeignore +++ b/.lycheeignore @@ -3,6 +3,9 @@ https://github\.com/.*/issues/? https://github\.com/.*/pull/? https://github\.com/.*/pull/[0-9]+ https://github\.com/.*/issues/[0-9]+ +https://github\.com/pingcap/tiflow/blob/master/dm/docs/.* +https://github\.com/\$user/(docs|docs-cn) +https://.*github.*/%7B%7B%7B%20.tidb_operator_version%20%7D%7D%7D https?://\$?\{host}/dashboard.* http://xn--\$?\{ip}-m86ht9t5l1bhz9ayu7b:3000.* http://ip:2379.* @@ -14,7 +17,6 @@ file://.*?http:/\$%7BPD_IP%7D:\$%7BPD_PORT%7D/dashboard.* http://\{grafana-ip\}:3000 http://\{pd-ip\}:2379/dashboard http://localhost:\d+/ -https://github\.com/\$user/(docs|docs-cn) https://linux\.die\.net/man.* https://dev\.mysql\.com/doc/.+/5.7/en/.* https://dev\.mysql\.com/doc/.+/8\.0/en/.* @@ -35,4 +37,4 @@ https://centminmod\.com/mydumper\.html https://docs\.pingcap\.com/tidb/v6\.6/system-variables#tidb_pessimistic_txn_aggressive_locking-new-in-v660 https://docs\.pingcap\.com/tidb/v7\.6/system-variables#tidb_ddl_version-new-in-v760 https://developers\.redhat\.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level -https://.*github.*/%7B%7B%7B%20.tidb_operator_version%20%7D%7D%7D +https://portal\.azure\.com/.* From b1c427a08ebff3188f57350320dcaf484e91fd3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Fri, 20 Mar 2026 07:26:14 +0100 Subject: [PATCH 5/5] Update dm/dm-precheck.md Co-authored-by: Airton Lastori --- dm/dm-precheck.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dm/dm-precheck.md b/dm/dm-precheck.md index 280a7cacecfab..e112a7b4cffb3 100644 --- a/dm/dm-precheck.md +++ b/dm/dm-precheck.md @@ -52,7 +52,7 @@ Regardless of the migration mode you choose, the precheck always includes the fo - Compatibility of the upstream MySQL table schema - - Check whether the upstream tables have foreign keys, which are not fully supported by TiDB. A warning is returned if a foreign key is found in the precheck. + - Check whether the upstream tables have foreign keys. DM does not replicate foreign key CASCADE operations (see [Incompatibility with foreign key CASCADE operations](/dm/dm-compatibility-catalog.md#incompatibility-with-foreign-key-cascade-operations)). A warning is returned if a foreign key is found in the precheck. - Check whether the upstream tables use character sets that are incompatible with TiDB. For more information, see [TiDB Supported Character Sets](/character-set-and-collation.md). - Check whether the upstream tables have primary key constraints or unique key constraints (introduced from v1.0.7).