From 8a06af6e1da226c110a3ee775a2576e3825f54df Mon Sep 17 00:00:00 2001 From: niuxianhui Date: Fri, 22 May 2026 23:04:08 -0700 Subject: [PATCH 1/2] [fix](doc) document json_extract no-auto-broadcast over arrays MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Issue #3318: a user upgrading from Doris 2.0.2 to 2.1.11 noticed the same SQL returned different results: `json_extract('', '$.k')` returned `[v1, v2, ...]` on 2.0.2 but `NULL` on 2.1.11. The 2.1+ behavior is intentional and engine-correct, aligning with MySQL's `JSON_EXTRACT` semantics: `$.k` traverses only object members and does not auto-broadcast over array elements. Verified in `be/src/util/jsonb_document.h::findValue` (MEMBER_CODE returns nullptr when pval is not an object) on `branch-2.1`, `branch-3.0`, `branch-3.1`, `branch-4.0`, `branch-4.1`. Doc did not previously call this out. Add a line to each of the 8 maintained version json-extract pages (current/2.1/3.x/4.x, EN + zh) explaining: - `$.k` on an array returns NULL — no auto-broadcast - Use `$[i].k` for index access - Array-wildcard broadcasting via `$[*].k` was introduced in 4.0 (the wildcard `continue` on 2.1/3.x does not actually iterate elements — verified by reading `findValue` on each release branch) - For 2.1/3.x, suggest `LATERAL VIEW EXPLODE` patterns for per-element extraction Closes #3318 --- .../scalar-functions/json-functions/json-extract.md | 1 + .../scalar-functions/json-functions/json-extract.md | 1 + .../scalar-functions/json-functions/json-extract.md | 2 ++ .../scalar-functions/json-functions/json-extract.md | 2 ++ .../scalar-functions/json-functions/json-extract.md | 1 + .../scalar-functions/json-functions/json-extract.md | 2 ++ .../scalar-functions/json-functions/json-extract.md | 2 ++ .../scalar-functions/json-functions/json-extract.md | 1 + 8 files changed, 12 insertions(+) diff --git a/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 1ffbda5be9971..01d6a0f20014b 100644 --- a/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -38,6 +38,7 @@ JSON_EXTRACT (, [, , ...]) * `*` represents a wildcard, where `$.*` represents all members of the root object, and `$[*]` represents all elements of the array. * `**` is used in combination with '$', '$**' represents all paths (including multi-level subpaths). - If `` contains wildcards (`*`), the matching results will be returned in array form. +- `` does not auto-broadcast over arrays. If `` is a JSON array and `` is `$.k`, the result is NULL — `$.k` only traverses object members. To target an element by index, use `$[i].k`; to extract a field from every element of an array, use the wildcard syntax `$[*].k`, which is supported from Doris 4.0 onward. ## Examples 1. General parameters diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 04bb633631d31..15310c5794eca 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -39,6 +39,7 @@ JSON_EXTRACT (, [, , ...]) * `**` 代表通配符，通常和 '$' 一起使用： '$**' 代表所有的路径（以及多层的子路径，见下面的示例 9）。 - 如果 `` 存在通配符(`*`)，匹配的结果会以数组形式返回。 +- `` 不会在数组上自动广播。如果 `` 是 JSON 数组，`` 为 `$.k`，结果为 NULL —— `$.k` 只会遍历 object 的成员。要按下标访问元素，使用 `$[i].k`；要从数组的每个元素中提取字段，使用通配符语法 `$[*].k`，该语法自 Doris 4.0 起支持。 ## 示例 1. 一般参数 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index f2faa89a3206b..ba75a7cebdfc8 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -67,6 +67,8 @@ JSON_EXTRACT_STRING (, ) * '[i]' 代表 json array 中下标为 i 的元素 - 获取 json_array 的最后一个元素可以用'$[last]'，倒数第二个元素可以用'$[last-1]'，以此类推。 +`` 不会在数组上自动广播：如果 JSON 值是数组、`` 为 `$.k`，结果为 NULL，因为 `$.k` 只会遍历 object 的成员。要按下标访问元素，使用 `$[i].k`。通过 `$[*].k` 在数组上广播取值是 Doris 4.0 引入的能力；在 2.1 和 3.x 上请改用 `LATERAL VIEW EXPLODE` 等方式逐元素展开后再 `json_extract`。 + ## 返回值 根据要提取的字段类型不同，返回目标 JSON 中 指定 JSON_PATH 的数据类型。特殊情况处理如下： * 如果 json_path 指定的字段在 JSON 中不存在，返回 NULL diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 3ac08098d0d44..d7ef025b6d3ca 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -76,6 +76,8 @@ JSON_EXTRACT_STRING (, ) * '[i]' 代表 json array 中下标为 i 的元素 - 获取 json_array 的最后一个元素可以用'$[last]'，倒数第二个元素可以用'$[last-1]'，以此类推。 +`` 不会在数组上自动广播：如果 JSON 值是数组、`` 为 `$.k`，结果为 NULL，因为 `$.k` 只会遍历 object 的成员。要按下标访问元素，使用 `$[i].k`。通过 `$[*].k` 在数组上广播取值是 Doris 4.0 引入的能力；在 2.1 和 3.x 上请改用 `LATERAL VIEW EXPLODE` 等方式逐元素展开后再 `json_extract`。 + ## 返回值 根据要提取的字段类型不同，返回目标 JSON 中 指定 JSON_PATH 的数据类型。特殊情况处理如下： * 如果 json_path 指定的字段在 JSON 中不存在，返回 NULL diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 04bb633631d31..15310c5794eca 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -39,6 +39,7 @@ JSON_EXTRACT (, [, , ...]) * `**` 代表通配符，通常和 '$' 一起使用： '$**' 代表所有的路径（以及多层的子路径，见下面的示例 9）。 - 如果 `` 存在通配符(`*`)，匹配的结果会以数组形式返回。 +- `` 不会在数组上自动广播。如果 `` 是 JSON 数组，`` 为 `$.k`，结果为 NULL —— `$.k` 只会遍历 object 的成员。要按下标访问元素，使用 `$[i].k`；要从数组的每个元素中提取字段，使用通配符语法 `$[*].k`，该语法自 Doris 4.0 起支持。 ## 示例 1. 一般参数 diff --git a/versioned_docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/versioned_docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index fa29fd7ded9f5..a7e4e981f88fc 100644 --- a/versioned_docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/versioned_docs/version-2.1/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -67,6 +67,8 @@ json path syntax: - '[i]' for element of json array at index i - Use '$[last]' to get the last element of json_array, and '$[last-1]' to get the penultimate element, and so on. +`` does not auto-broadcast over arrays: if the JSON value is an array and `` is `$.k`, the result is NULL because `$.k` only traverses object members. To target an element by index use `$[i].k`. Array-wildcard broadcasting via `$[*].k` was introduced in Doris 4.0; on 2.1 and 3.x, extract per-element values via `LATERAL VIEW EXPLODE` patterns instead. + ## Return Values According to the type of the field to be extracted, return the data type of the specified JSON_PATH in the target JSON. Special case handling is as follows: * If the field specified by json_path does not exist in the JSON, return NULL. diff --git a/versioned_docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/versioned_docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index b3ad42edbeaee..eaa6f4afe760e 100644 --- a/versioned_docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/versioned_docs/version-3.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -76,6 +76,8 @@ json path syntax: - '[i]' for element of json array at index i - Use '$[last]' to get the last element of json_array, and '$[last-1]' to get the penultimate element, and so on. +`` does not auto-broadcast over arrays: if the JSON value is an array and `` is `$.k`, the result is NULL because `$.k` only traverses object members. To target an element by index use `$[i].k`. Array-wildcard broadcasting via `$[*].k` was introduced in Doris 4.0; on 2.1 and 3.x, extract per-element values via `LATERAL VIEW EXPLODE` patterns instead. + ## Return Values According to the type of the field to be extracted, return the data type of the specified JSON_PATH in the target JSON. Special case handling is as follows: * If the field specified by json_path does not exist in the JSON, return NULL. diff --git a/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index dd9615a25c056..e71f84669aa69 100644 --- a/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -38,6 +38,7 @@ JSON_EXTRACT (, [, , ...]) * `*` represents a wildcard, where `$.*` represents all members of the root object, and `$[*]` represents all elements of the array. * `**` is used in combination with '$', '$**' represents all paths (including multi-level subpaths). - If `` contains wildcards (`*`), the matching results will be returned in array form. +- `` does not auto-broadcast over arrays. If `` is a JSON array and `` is `$.k`, the result is NULL — `$.k` only traverses object members. To target an element by index, use `$[i].k`; to extract a field from every element of an array, use the wildcard syntax `$[*].k`, which is supported from Doris 4.0 onward. ## Examples 1. General parameters From d21e45776630f03b0dfa31e94dd95b946e7be5ae Mon Sep 17 00:00:00 2001 From: niuxianhui Date: Fri, 22 May 2026 23:18:01 -0700 Subject: [PATCH 2/2] [doc] add $[*].k positive example to json_extract (4.x and current) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Issue #3318 reports that json_extract returns NULL when $.k is applied over a JSON array, with no clear path to the broadcasted result the user wants. The existing description note explains $[*].k as the 4.0+ solution but provides no runnable example. Add example 11 to the current and 4.x docs (EN + zh) contrasting $.k (NULL) and $[*].k (broadcasted array). Outputs verified against a 4.x cluster. Not added to 2.1/3.x docs since $[*].k returns NULL there too — the description already notes "supported from Doris 4.0 onward". Co-Authored-By: Claude Opus 4.7 (1M context) --- .../json-functions/json-extract.md | 24 +++++++++++++++++++ .../json-functions/json-extract.md | 24 +++++++++++++++++++ .../json-functions/json-extract.md | 24 +++++++++++++++++++ .../json-functions/json-extract.md | 24 +++++++++++++++++++ 4 files changed, 96 insertions(+) diff --git a/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 01d6a0f20014b..c7e9b59e30c56 100644 --- a/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/docs/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -186,3 +186,27 @@ JSON_EXTRACT (, [, , ...]) | null | 0 | +------+------+ ``` + +11. Extracting a field from each element of a JSON array (Doris 4.0+) + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k'); + ``` + ``` + +--------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k') | + +--------------------------------------------------+ + | NULL | + +--------------------------------------------------+ + ``` + > `$.k` does not traverse arrays; it only descends into object members. + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k'); + ``` + ``` + +-----------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k') | + +-----------------------------------------------------+ + | [1,2,3] | + +-----------------------------------------------------+ + ``` + > Use `$[*].k` to extract the field from every element of the array. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 15310c5794eca..d0296536e5bf7 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -186,3 +186,27 @@ JSON_EXTRACT (, [, , ...]) | null | 0 | +------+------+ ``` + +11. 从 JSON 数组的每个元素中提取字段（Doris 4.0+） + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k'); + ``` + ``` + +--------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k') | + +--------------------------------------------------+ + | NULL | + +--------------------------------------------------+ + ``` + > `$.k` 不会遍历数组，只会下钻到对象的成员。 + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k'); + ``` + ``` + +-----------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k') | + +-----------------------------------------------------+ + | [1,2,3] | + +-----------------------------------------------------+ + ``` + > 使用 `$[*].k` 从数组的每个元素提取字段。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index 15310c5794eca..d0296536e5bf7 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -186,3 +186,27 @@ JSON_EXTRACT (, [, , ...]) | null | 0 | +------+------+ ``` + +11. 从 JSON 数组的每个元素中提取字段（Doris 4.0+） + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k'); + ``` + ``` + +--------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k') | + +--------------------------------------------------+ + | NULL | + +--------------------------------------------------+ + ``` + > `$.k` 不会遍历数组，只会下钻到对象的成员。 + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k'); + ``` + ``` + +-----------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k') | + +-----------------------------------------------------+ + | [1,2,3] | + +-----------------------------------------------------+ + ``` + > 使用 `$[*].k` 从数组的每个元素提取字段。 diff --git a/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md b/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md index e71f84669aa69..3f249ddbda67c 100644 --- a/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md +++ b/versioned_docs/version-4.x/sql-manual/sql-functions/scalar-functions/json-functions/json-extract.md @@ -186,3 +186,27 @@ JSON_EXTRACT (, [, , ...]) | null | 0 | +------+------+ ``` + +11. Extracting a field from each element of a JSON array (Doris 4.0+) + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k'); + ``` + ``` + +--------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$.k') | + +--------------------------------------------------+ + | NULL | + +--------------------------------------------------+ + ``` + > `$.k` does not traverse arrays; it only descends into object members. + ```sql + select json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k'); + ``` + ``` + +-----------------------------------------------------+ + | json_extract('[{"k":1},{"k":2},{"k":3}]', '$[*].k') | + +-----------------------------------------------------+ + | [1,2,3] | + +-----------------------------------------------------+ + ``` + > Use `$[*].k` to extract the field from every element of the array.