Skip to content
1 change: 1 addition & 0 deletions TOC-tidb-cloud-essential.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@
- Attributes
- [AUTO_INCREMENT](/auto-increment.md)
- [AUTO_RANDOM](/auto-random.md)
- [_tidb_rowid](/tidb-rowid.md)
- [SHARD_ROW_ID_BITS](/shard-row-id-bits.md)
- [Literal Values](/literal-values.md)
- [Schema Object Names](/schema-object-names.md)
Expand Down
1 change: 1 addition & 0 deletions TOC-tidb-cloud-premium.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@
- Attributes
- [AUTO_INCREMENT](/auto-increment.md)
- [AUTO_RANDOM](/auto-random.md)
- [_tidb_rowid](/tidb-rowid.md)
- [SHARD_ROW_ID_BITS](/shard-row-id-bits.md)
- [Literal Values](/literal-values.md)
- [Schema Object Names](/schema-object-names.md)
Expand Down
1 change: 1 addition & 0 deletions TOC-tidb-cloud-starter.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@
- Attributes
- [AUTO_INCREMENT](/auto-increment.md)
- [AUTO_RANDOM](/auto-random.md)
- [_tidb_rowid](/tidb-rowid.md)
- [SHARD_ROW_ID_BITS](/shard-row-id-bits.md)
- [Literal Values](/literal-values.md)
- [Schema Object Names](/schema-object-names.md)
Expand Down
1 change: 1 addition & 0 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,7 @@
- Attributes
- [AUTO_INCREMENT](/auto-increment.md)
- [AUTO_RANDOM](/auto-random.md)
- [_tidb_rowid](/tidb-rowid.md)
- [SHARD_ROW_ID_BITS](/shard-row-id-bits.md)
- [Literal Values](/literal-values.md)
- [Schema Object Names](/schema-object-names.md)
Expand Down
1 change: 1 addition & 0 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -636,6 +636,7 @@
- Attributes
- [AUTO_INCREMENT](/auto-increment.md)
- [AUTO_RANDOM](/auto-random.md)
- [_tidb_rowid](/tidb-rowid.md)
- [SHARD_ROW_ID_BITS](/shard-row-id-bits.md)
- [Literal Values](/literal-values.md)
- [Schema Object Names](/schema-object-names.md)
Expand Down
2 changes: 1 addition & 1 deletion clustered-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The term _clustered_ in this context refers to the _organization of how data is

Currently, tables containing primary keys in TiDB are divided into the following two categories:

- `NONCLUSTERED`: The primary key of the table is non-clustered index. In tables with non-clustered indexes, the keys for row data consist of internal `_tidb_rowid` implicitly assigned by TiDB. Because primary keys are essentially unique indexes, tables with non-clustered indexes need at least two key-value pairs to store a row, which are:
- `NONCLUSTERED`: The primary key of the table is non-clustered index. In tables with non-clustered indexes, the keys for row data consist of internal [`_tidb_rowid`](/tidb-rowid.md) values implicitly assigned by TiDB. Because primary keys are essentially unique indexes, tables with non-clustered indexes need at least two key-value pairs to store a row, which are:
- `_tidb_rowid` (key) - row data (value)
- Primary key data (key) - `_tidb_rowid` (value)
- `CLUSTERED`: The primary key of the table is clustered index. In tables with clustered indexes, the keys for row data consist of primary key data given by the user. Therefore, tables with clustered indexes need only one key-value pair to store a row, which is:
Expand Down
10 changes: 7 additions & 3 deletions shard-row-id-bits.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@

# SHARD_ROW_ID_BITS

This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the shards after the implicit `_tidb_rowid` is sharded.
This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the shards after the implicit [`_tidb_rowid`](/tidb-rowid.md) is sharded.

## Concept

For the tables with a non-clustered primary key or no primary key, TiDB uses an implicit auto-increment row ID. When a large number of `INSERT` operations are performed, the data is written into a single Region, causing a write hot spot.
For tables with a non-clustered primary key or no primary key, TiDB uses the hidden [`_tidb_rowid`](/tidb-rowid.md) as an implicit auto-increment row ID. When a large number of `INSERT` operations are performed, the data is written into a single Region, causing a write hot spot.

Check warning on line 12 in shard-row-id-bits.md

View workflow job for this annotation

GitHub Actions / vale

[vale] reported by reviewdog 🐶 [PingCAP.Ambiguous] Consider using a clearer word than 'a large number of' because it may cause confusion. Raw Output: {"message": "[PingCAP.Ambiguous] Consider using a clearer word than 'a large number of' because it may cause confusion.", "location": {"path": "shard-row-id-bits.md", "range": {"start": {"line": 12, "column": 160}}}, "severity": "INFO"}

To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The row IDs are scattered and the data are written into multiple different Regions.

Expand All @@ -23,9 +23,13 @@
|--------|--------|--------------|
| 1 bit | `S` bits | `63-S` bits |

- The values of the auto-increment bits are stored in TiKV and allocated sequentially. Each time a value is allocated, the next value is incremented by 1. The auto-increment bits ensure that the column values of `_tidb_rowid` are unique globally. When the value of the auto-increment bits is exhausted (that is, when the maximum value is reached), subsequent automatic allocations fail with the error `Failed to read auto-increment value from storage engine`.
- The values of the auto-increment bits are stored in TiKV and allocated sequentially. Each time a value is allocated, the next value is incremented by 1. When the value of the auto-increment bits is exhausted (that is, when the maximum value is reached), subsequent automatic allocations fail with the error `Failed to read auto-increment value from storage engine`.
- The value range of `_tidb_rowid`: the maximum number of bits for the final generated value = shard bits + auto-increment bits, so the maximum value is `(2^63)-1`.

> **Warning:**
>
> `_tidb_rowid` is an internal row handle. Do not assume it is globally unique in all cases. For partitioned tables that do not use clustered indexes, `ALTER TABLE ... EXCHANGE PARTITION` can leave different partitions with the same `_tidb_rowid` value. For details, see [`_tidb_rowid`](/tidb-rowid.md).

> **Note:**
>
> Selection of shard bits (`S`):
Expand Down
3 changes: 2 additions & 1 deletion sql-statements/sql-statement-show-table-next-rowid.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aliases: ['/docs/dev/sql-statements/sql-statement-show-table-next-rowid/']

`SHOW TABLE NEXT_ROW_ID` is used to show the details of some special columns of a table, including:

* [`AUTO_INCREMENT`](/auto-increment.md) column automatically created by TiDB, namely, `_tidb_rowid` column.
* [`_tidb_rowid`](/tidb-rowid.md), the hidden row-handle column automatically managed by TiDB for supported tables.
* `AUTO_INCREMENT` column created by users.
* [`AUTO_RANDOM`](/auto-random.md) column created by users.
* [`SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) created by users.
Expand Down Expand Up @@ -66,3 +66,4 @@ This statement is a TiDB extension to MySQL syntax.
* [CREATE TABLE](/sql-statements/sql-statement-create-table.md)
* [AUTO_RANDOM](/auto-random.md)
* [CREATE_SEQUENCE](/sql-statements/sql-statement-create-sequence.md)
* [_tidb_rowid](/tidb-rowid.md)
160 changes: 160 additions & 0 deletions tidb-rowid.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
---
title: _tidb_rowid
summary: Learn what `_tidb_rowid` is, when it is available, and how to use it safely.
---

# _tidb_rowid

`_tidb_rowid` is a hidden system column that TiDB uses as the row handle for tables that do not use a clustered index. You cannot declare this column in the table schema, but you can reference it in SQL when the table uses `_tidb_rowid` as its handle.

In the current implementation, `_tidb_rowid` is an extra `BIGINT NOT NULL` handle column managed by TiDB.

> **Warning:**
>
> - Do not assume `_tidb_rowid` is globally unique in all cases. For partitioned tables that do not use clustered indexes, executing `ALTER TABLE ... EXCHANGE PARTITION` can leave different partitions with the same `_tidb_rowid` value.
> - If you need a stable unique identifier, define and use an explicit primary key instead of relying on `_tidb_rowid`.

## When `_tidb_rowid` is available

`_tidb_rowid` is available for tables whose row handle is not a clustered primary key. In practice, this means the following table types use `_tidb_rowid`:

- Tables without primary keys
- Tables with primary keys that are explicitly defined as `NONCLUSTERED`

`_tidb_rowid` is not available for tables that use a clustered index, including the following:

- Tables with integer primary keys that are clustered row handles
- Tables with clustered indexes on composite primary keys

The following example shows the difference:

```sql
CREATE TABLE t1 (a INT, b VARCHAR(20));
CREATE TABLE t2 (id BIGINT PRIMARY KEY NONCLUSTERED, a INT);
CREATE TABLE t3 (id BIGINT PRIMARY KEY CLUSTERED, a INT);
```

For `t1` and `t2`, you can query `_tidb_rowid`:

```sql
SELECT _tidb_rowid, a, b FROM t1;
SELECT _tidb_rowid, id, a FROM t2;
```

For `t3`, `_tidb_rowid` is unavailable because the clustered primary key is already the row handle:

```sql
SELECT _tidb_rowid, id, a FROM t3;
```

```sql
ERROR 1054 (42S22): Unknown column '_tidb_rowid' in 'field list'
```

## Read `_tidb_rowid`

You can use `_tidb_rowid` in `SELECT` statements for supported tables. This is useful for tasks such as pagination, troubleshooting, and batch processing.

Example:

```sql
CREATE TABLE t (a INT, b VARCHAR(20));
INSERT INTO t VALUES (1, 'x'), (2, 'y');

SELECT _tidb_rowid, a, b FROM t ORDER BY _tidb_rowid;
```

```sql
+-------------+---+---+
| _tidb_rowid | a | b |
+-------------+---+---+
| 1 | 1 | x |
| 2 | 2 | y |
+-------------+---+---+
```

To inspect the next value that TiDB will allocate for the row ID, use `SHOW TABLE ... NEXT_ROW_ID`:

```sql
SHOW TABLE t NEXT_ROW_ID;
```

```sql
+-----------------------+------------+-------------+--------------------+-------------+
| DB_NAME | TABLE_NAME | COLUMN_NAME | NEXT_GLOBAL_ROW_ID | ID_TYPE |
+-----------------------+------------+-------------+--------------------+-------------+
| update_doc_rowid_test | t | _tidb_rowid | 30001 | _TIDB_ROWID |
+-----------------------+------------+-------------+--------------------+-------------+
```

## Write `_tidb_rowid`

By default, TiDB does not allow `INSERT`, `REPLACE`, or `UPDATE` statements to write `_tidb_rowid` directly.

```sql
INSERT INTO t(_tidb_rowid, a, b) VALUES (101, 4, 'w');
```

```sql
ERROR 1105 (HY000): insert, update and replace statements for _tidb_rowid are not supported
```

If you need to preserve row IDs during data import or migration, enable the system variable [`tidb_opt_write_row_id`](/system-variables.md#tidb_opt_write_row_id) first:

```sql
SET @@tidb_opt_write_row_id = ON;
INSERT INTO t(_tidb_rowid, a, b) VALUES (100, 3, 'z');
SET @@tidb_opt_write_row_id = OFF;

SELECT _tidb_rowid, a, b FROM t WHERE _tidb_rowid = 100;
```

```sql
+-------------+---+---+
| _tidb_rowid | a | b |
+-------------+---+---+
| 100 | 3 | z |
+-------------+---+---+
```

> **Warning:**
>
> `tidb_opt_write_row_id` is intended for import and migration scenarios. It is not recommended for regular application writes.

## Restrictions

- You cannot create a user column named `_tidb_rowid`.
- You cannot rename an existing user column to `_tidb_rowid`.
- `_tidb_rowid` is an internal row handle. Do not treat it as a long-term business key.
- On partitioned non-clustered tables, `_tidb_rowid` values are not guaranteed to be unique across partitions. After you execute `EXCHANGE PARTITION`, different partitions can contain rows with the same `_tidb_rowid` value.
- Whether `_tidb_rowid` exists depends on the table layout. If a table uses a clustered index, use the primary key instead.

## Hotspot considerations

For tables that use `_tidb_rowid`, TiDB allocates row IDs in increasing order by default. In write-intensive workloads, this can create write hotspots.

To mitigate this issue for tables that rely on implicit row IDs, consider using [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md) and, if needed, [`PRE_SPLIT_REGIONS`](/sql-statements/sql-statement-split-region.md#pre_split_regions).

Example:

```sql
CREATE TABLE t (
id BIGINT PRIMARY KEY NONCLUSTERED,
c INT
) SHARD_ROW_ID_BITS = 4;
```

`SHARD_ROW_ID_BITS` applies only to tables that use the implicit row ID path. It does not apply to clustered-index tables.

## Related statements and variables

- [`SHOW TABLE NEXT_ROW_ID`](/sql-statements/sql-statement-show-table-next-rowid.md): shows the next row ID that TiDB will allocate
- [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md): shards implicit row IDs to reduce hotspots
- [`Clustered Indexes`](/clustered-indexes.md): explains when a table uses the primary key instead of `_tidb_rowid`
- [`tidb_opt_write_row_id`](/system-variables.md#tidb_opt_write_row_id): controls whether writes to `_tidb_rowid` are allowed

## See also

- [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md)
- [`AUTO_INCREMENT`](/auto-increment.md)
- [Non-transactional DML](/non-transactional-dml.md)
Loading