diff --git a/TOC-tidb-cloud-essential.md b/TOC-tidb-cloud-essential.md index cdeb3c7ca755d..2566c94ad9424 100644 --- a/TOC-tidb-cloud-essential.md +++ b/TOC-tidb-cloud-essential.md @@ -202,6 +202,7 @@ - Attributes - [AUTO_INCREMENT](/auto-increment.md) - [AUTO_RANDOM](/auto-random.md) + - [_tidb_rowid](/tidb-rowid.md) - [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) - [Literal Values](/literal-values.md) - [Schema Object Names](/schema-object-names.md) diff --git a/TOC-tidb-cloud-premium.md b/TOC-tidb-cloud-premium.md index 299577ac3e16e..85feb6df36c84 100644 --- a/TOC-tidb-cloud-premium.md +++ b/TOC-tidb-cloud-premium.md @@ -281,6 +281,7 @@ - Attributes - [AUTO_INCREMENT](/auto-increment.md) - [AUTO_RANDOM](/auto-random.md) + - [_tidb_rowid](/tidb-rowid.md) - [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) - [Literal Values](/literal-values.md) - [Schema Object Names](/schema-object-names.md) diff --git a/TOC-tidb-cloud-starter.md b/TOC-tidb-cloud-starter.md index d8fa6020884bd..4d0ced924dcff 100644 --- a/TOC-tidb-cloud-starter.md +++ b/TOC-tidb-cloud-starter.md @@ -208,6 +208,7 @@ - Attributes - [AUTO_INCREMENT](/auto-increment.md) - [AUTO_RANDOM](/auto-random.md) + - [_tidb_rowid](/tidb-rowid.md) - [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) - [Literal Values](/literal-values.md) - [Schema Object Names](/schema-object-names.md) diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index 2c0959c954a6b..7b97ca11c7a6c 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -268,6 +268,7 @@ - Attributes - [AUTO_INCREMENT](/auto-increment.md) - [AUTO_RANDOM](/auto-random.md) + - [_tidb_rowid](/tidb-rowid.md) - [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) - [Literal Values](/literal-values.md) - [Schema Object Names](/schema-object-names.md) diff --git a/TOC.md b/TOC.md index c772134267106..1aa777cd68227 100644 --- a/TOC.md +++ b/TOC.md @@ -636,6 +636,7 @@ - Attributes - [AUTO_INCREMENT](/auto-increment.md) - [AUTO_RANDOM](/auto-random.md) + - [_tidb_rowid](/tidb-rowid.md) - [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) - [Literal Values](/literal-values.md) - [Schema Object Names](/schema-object-names.md) diff --git a/clustered-indexes.md b/clustered-indexes.md index 7b35047e245c8..8e474debdd345 100644 --- a/clustered-indexes.md +++ b/clustered-indexes.md @@ -11,7 +11,7 @@ The term _clustered_ in this context refers to the _organization of how data is Currently, tables containing primary keys in TiDB are divided into the following two categories: -- `NONCLUSTERED`: The primary key of the table is non-clustered index. In tables with non-clustered indexes, the keys for row data consist of internal `_tidb_rowid` implicitly assigned by TiDB. Because primary keys are essentially unique indexes, tables with non-clustered indexes need at least two key-value pairs to store a row, which are: +- `NONCLUSTERED`: The primary key of the table is non-clustered index. In tables with non-clustered indexes, the keys for row data consist of internal [`_tidb_rowid`](/tidb-rowid.md) values implicitly assigned by TiDB. Because primary keys are essentially unique indexes, tables with non-clustered indexes need at least two key-value pairs to store a row, which are: - `_tidb_rowid` (key) - row data (value) - Primary key data (key) - `_tidb_rowid` (value) - `CLUSTERED`: The primary key of the table is clustered index. In tables with clustered indexes, the keys for row data consist of primary key data given by the user. Therefore, tables with clustered indexes need only one key-value pair to store a row, which is: diff --git a/shard-row-id-bits.md b/shard-row-id-bits.md index 38073c00b5da2..7a5cf64ffcce8 100644 --- a/shard-row-id-bits.md +++ b/shard-row-id-bits.md @@ -5,11 +5,11 @@ summary: Learn the SHARD_ROW_ID_BITS attribute. # SHARD_ROW_ID_BITS -This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the shards after the implicit `_tidb_rowid` is sharded. +This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the shards after the implicit [`_tidb_rowid`](/tidb-rowid.md) is sharded. ## Concept -For the tables with a non-clustered primary key or no primary key, TiDB uses an implicit auto-increment row ID. When a large number of `INSERT` operations are performed, the data is written into a single Region, causing a write hot spot. +For tables with a non-clustered primary key or no primary key, TiDB uses the hidden [`_tidb_rowid`](/tidb-rowid.md) as an implicit auto-increment row ID. When a large number of `INSERT` operations are performed, the data is written into a single Region, causing a write hot spot. To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The row IDs are scattered and the data are written into multiple different Regions. @@ -23,9 +23,13 @@ When you set `SHARD_ROW_ID_BITS = S`, the structure of `_tidb_rowid` is as follo |--------|--------|--------------| | 1 bit | `S` bits | `63-S` bits | -- The values of the auto-increment bits are stored in TiKV and allocated sequentially. Each time a value is allocated, the next value is incremented by 1. The auto-increment bits ensure that the column values of `_tidb_rowid` are unique globally. When the value of the auto-increment bits is exhausted (that is, when the maximum value is reached), subsequent automatic allocations fail with the error `Failed to read auto-increment value from storage engine`. +- The values of the auto-increment bits are stored in TiKV and allocated sequentially. Each time a value is allocated, the next value is incremented by 1. When the value of the auto-increment bits is exhausted (that is, when the maximum value is reached), subsequent automatic allocations fail with the error `Failed to read auto-increment value from storage engine`. - The value range of `_tidb_rowid`: the maximum number of bits for the final generated value = shard bits + auto-increment bits, so the maximum value is `(2^63)-1`. +> **Warning:** +> +> `_tidb_rowid` is an internal row handle. Do not assume it is globally unique in all cases. For partitioned tables that do not use clustered indexes, `ALTER TABLE ... EXCHANGE PARTITION` can leave different partitions with the same `_tidb_rowid` value. For details, see [`_tidb_rowid`](/tidb-rowid.md). + > **Note:** > > Selection of shard bits (`S`): diff --git a/sql-statements/sql-statement-show-table-next-rowid.md b/sql-statements/sql-statement-show-table-next-rowid.md index 73ee64ae0fb49..59766dbff7403 100644 --- a/sql-statements/sql-statement-show-table-next-rowid.md +++ b/sql-statements/sql-statement-show-table-next-rowid.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/sql-statements/sql-statement-show-table-next-rowid/'] `SHOW TABLE NEXT_ROW_ID` is used to show the details of some special columns of a table, including: -* [`AUTO_INCREMENT`](/auto-increment.md) column automatically created by TiDB, namely, `_tidb_rowid` column. +* [`_tidb_rowid`](/tidb-rowid.md), the hidden row-handle column automatically managed by TiDB for supported tables. * `AUTO_INCREMENT` column created by users. * [`AUTO_RANDOM`](/auto-random.md) column created by users. * [`SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) created by users. @@ -66,3 +66,4 @@ This statement is a TiDB extension to MySQL syntax. * [CREATE TABLE](/sql-statements/sql-statement-create-table.md) * [AUTO_RANDOM](/auto-random.md) * [CREATE_SEQUENCE](/sql-statements/sql-statement-create-sequence.md) +* [_tidb_rowid](/tidb-rowid.md) diff --git a/tidb-rowid.md b/tidb-rowid.md new file mode 100644 index 0000000000000..320edf2be3903 --- /dev/null +++ b/tidb-rowid.md @@ -0,0 +1,160 @@ +--- +title: _tidb_rowid +summary: Learn what `_tidb_rowid` is, when it is available, and how to use it safely. +--- + +# _tidb_rowid + +`_tidb_rowid` is a hidden system column that TiDB uses as the row handle for tables that do not use a clustered index. You cannot declare this column in the table schema, but you can reference it in SQL when the table uses `_tidb_rowid` as its handle. + +In the current implementation, `_tidb_rowid` is an extra `BIGINT NOT NULL` handle column managed by TiDB. + +> **Warning:** +> +> - Do not assume `_tidb_rowid` is globally unique in all cases. For partitioned tables that do not use clustered indexes, executing `ALTER TABLE ... EXCHANGE PARTITION` can leave different partitions with the same `_tidb_rowid` value. +> - If you need a stable unique identifier, define and use an explicit primary key instead of relying on `_tidb_rowid`. + +## When `_tidb_rowid` is available + +`_tidb_rowid` is available for tables whose row handle is not a clustered primary key. In practice, this means the following table types use `_tidb_rowid`: + +- Tables without primary keys +- Tables with primary keys that are explicitly defined as `NONCLUSTERED` + +`_tidb_rowid` is not available for tables that use a clustered index, including the following: + +- Tables with integer primary keys that are clustered row handles +- Tables with clustered indexes on composite primary keys + +The following example shows the difference: + +```sql +CREATE TABLE t1 (a INT, b VARCHAR(20)); +CREATE TABLE t2 (id BIGINT PRIMARY KEY NONCLUSTERED, a INT); +CREATE TABLE t3 (id BIGINT PRIMARY KEY CLUSTERED, a INT); +``` + +For `t1` and `t2`, you can query `_tidb_rowid`: + +```sql +SELECT _tidb_rowid, a, b FROM t1; +SELECT _tidb_rowid, id, a FROM t2; +``` + +For `t3`, `_tidb_rowid` is unavailable because the clustered primary key is already the row handle: + +```sql +SELECT _tidb_rowid, id, a FROM t3; +``` + +```sql +ERROR 1054 (42S22): Unknown column '_tidb_rowid' in 'field list' +``` + +## Read `_tidb_rowid` + +You can use `_tidb_rowid` in `SELECT` statements for supported tables. This is useful for tasks such as pagination, troubleshooting, and batch processing. + +Example: + +```sql +CREATE TABLE t (a INT, b VARCHAR(20)); +INSERT INTO t VALUES (1, 'x'), (2, 'y'); + +SELECT _tidb_rowid, a, b FROM t ORDER BY _tidb_rowid; +``` + +```sql ++-------------+---+---+ +| _tidb_rowid | a | b | ++-------------+---+---+ +| 1 | 1 | x | +| 2 | 2 | y | ++-------------+---+---+ +``` + +To inspect the next value that TiDB will allocate for the row ID, use `SHOW TABLE ... NEXT_ROW_ID`: + +```sql +SHOW TABLE t NEXT_ROW_ID; +``` + +```sql ++-----------------------+------------+-------------+--------------------+-------------+ +| DB_NAME | TABLE_NAME | COLUMN_NAME | NEXT_GLOBAL_ROW_ID | ID_TYPE | ++-----------------------+------------+-------------+--------------------+-------------+ +| update_doc_rowid_test | t | _tidb_rowid | 30001 | _TIDB_ROWID | ++-----------------------+------------+-------------+--------------------+-------------+ +``` + +## Write `_tidb_rowid` + +By default, TiDB does not allow `INSERT`, `REPLACE`, or `UPDATE` statements to write `_tidb_rowid` directly. + +```sql +INSERT INTO t(_tidb_rowid, a, b) VALUES (101, 4, 'w'); +``` + +```sql +ERROR 1105 (HY000): insert, update and replace statements for _tidb_rowid are not supported +``` + +If you need to preserve row IDs during data import or migration, enable the system variable [`tidb_opt_write_row_id`](/system-variables.md#tidb_opt_write_row_id) first: + +```sql +SET @@tidb_opt_write_row_id = ON; +INSERT INTO t(_tidb_rowid, a, b) VALUES (100, 3, 'z'); +SET @@tidb_opt_write_row_id = OFF; + +SELECT _tidb_rowid, a, b FROM t WHERE _tidb_rowid = 100; +``` + +```sql ++-------------+---+---+ +| _tidb_rowid | a | b | ++-------------+---+---+ +| 100 | 3 | z | ++-------------+---+---+ +``` + +> **Warning:** +> +> `tidb_opt_write_row_id` is intended for import and migration scenarios. It is not recommended for regular application writes. + +## Restrictions + +- You cannot create a user column named `_tidb_rowid`. +- You cannot rename an existing user column to `_tidb_rowid`. +- `_tidb_rowid` is an internal row handle. Do not treat it as a long-term business key. +- On partitioned non-clustered tables, `_tidb_rowid` values are not guaranteed to be unique across partitions. After you execute `EXCHANGE PARTITION`, different partitions can contain rows with the same `_tidb_rowid` value. +- Whether `_tidb_rowid` exists depends on the table layout. If a table uses a clustered index, use the primary key instead. + +## Hotspot considerations + +For tables that use `_tidb_rowid`, TiDB allocates row IDs in increasing order by default. In write-intensive workloads, this can create write hotspots. + +To mitigate this issue for tables that rely on implicit row IDs, consider using [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md) and, if needed, [`PRE_SPLIT_REGIONS`](/sql-statements/sql-statement-split-region.md#pre_split_regions). + +Example: + +```sql +CREATE TABLE t ( + id BIGINT PRIMARY KEY NONCLUSTERED, + c INT +) SHARD_ROW_ID_BITS = 4; +``` + +`SHARD_ROW_ID_BITS` applies only to tables that use the implicit row ID path. It does not apply to clustered-index tables. + +## Related statements and variables + +- [`SHOW TABLE NEXT_ROW_ID`](/sql-statements/sql-statement-show-table-next-rowid.md): shows the next row ID that TiDB will allocate +- [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md): shards implicit row IDs to reduce hotspots +- [`Clustered Indexes`](/clustered-indexes.md): explains when a table uses the primary key instead of `_tidb_rowid` +- [`tidb_opt_write_row_id`](/system-variables.md#tidb_opt_write_row_id): controls whether writes to `_tidb_rowid` are allowed + +## See also + +- [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) +- [`AUTO_INCREMENT`](/auto-increment.md) +- [Non-transactional DML](/non-transactional-dml.md)