Add columnar index mode overview - #7594
Conversation
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
Elastic Docs Style Checker (Vale)Summary: 3 suggestions found 💡 Suggestions (3): Optional style improvements. Apply when helpful.
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
|
|
||
| # Columnar index mode [columnar-index-mode] | ||
|
|
||
| Columnar index mode turns {{es}} into an analytical and search columnar store for the indices where you enable it. Instead of keeping multiple copies of each field for different query paths, columnar mode stores fields once as [doc values](elasticsearch://reference/elasticsearch/mapping-reference/doc-values.md) by default, which can reduce storage cost for high-volume, analytics-heavy data while keeping the same APIs, dashboards, and integrations. |
There was a problem hiding this comment.
| Columnar index mode turns {{es}} into an analytical and search columnar store for the indices where you enable it. Instead of keeping multiple copies of each field for different query paths, columnar mode stores fields once as [doc values](elasticsearch://reference/elasticsearch/mapping-reference/doc-values.md) by default, which can reduce storage cost for high-volume, analytics-heavy data while keeping the same APIs, dashboards, and integrations. | |
| Columnar index mode turns {{es}} into an analytical and search columnar store for the indices where you enable it. Instead of keeping multiple copies of each field for different query paths, columnar mode stores fields once as [doc values](elasticsearch://reference/elasticsearch/mapping-reference/doc-values.md) by default. This helps reduce storage cost for high-volume, analytics-heavy data while keeping the same APIs, dashboards, and integrations. |
| * Operational and business analytics: Dashboards and aggregations over application events, transactions, or IoT readings that would otherwise push you to a separate analytics store. | ||
|
|
||
| Columnar mode isn't a universal default. | ||
| Prefer a standard index (or another specialized mode) when your workload is document-centric search that needs the original JSON `_source` as submitted, or when you rely on inverted indexes and related structures on most fields by default. |
There was a problem hiding this comment.
| Prefer a standard index (or another specialized mode) when your workload is document-centric search that needs the original JSON `_source` as submitted, or when you rely on inverted indexes and related structures on most fields by default. | |
| Prefer a `standard` index (or another specialized mode) when your workload is document-centric search that needs the original JSON `_source` as submitted, or when you rely on inverted indexes and related structures on most fields by default. |
There was a problem hiding this comment.
Adding line break and putting "standard" in backticks to match the format used earlier on the page
| Prefer a standard index (or another specialized mode) when your workload is document-centric search that needs the original JSON `_source` as submitted, or when you rely on inverted indexes and related structures on most fields by default. | ||
|
|
||
| For metrics that need time series dimensions and metric field semantics, use a [time series data stream](/manage-data/data-store/data-streams/time-series-data-stream-tsds.md) instead. | ||
| For logs that should keep the current `logsdb` defaults without switching to a fully columnar store, use a [logs data stream](/manage-data/data-store/data-streams/logs-data-stream.md). |
There was a problem hiding this comment.
I wonder if all this info about use cases would work better in a table. Might be easier to reference which mode to use when
|
|
||
| :::{note} | ||
| :applies_to: {"stack": "preview 9.5", "serverless": "preview"} | ||
| For a fully columnar storage profile for logs, you can set `index.mode` to `logsdb_columnar` instead. For when to choose that mode, refer to [Columnar index mode](/manage-data/data-store/columnar.md). |
There was a problem hiding this comment.
| For a fully columnar storage profile for logs, you can set `index.mode` to `logsdb_columnar` instead. For when to choose that mode, refer to [Columnar index mode](/manage-data/data-store/columnar.md). | |
| For a fully columnar logs storage profile, you can set `index.mode` to `logsdb_columnar` instead. To learn when to choose that mode, refer to [](/manage-data/data-store/columnar.md). |
Just a suggestion to avoid reusing "for" ;) + removing link text since it's the same as the page title, so docsbuilder should auto-populate it (easier maintenance)
Summary
Relates to elastic/roadmap#290
This PR adds an introductory overview of the new columnar index mode (and small links and comments in the related pages in the "Manage data" section of the documentation).
NOTE: Unlike the existing logsdb page (https://www.elastic.co/docs/manage-data/data-store/data-streams/logs-data-stream), this overview does not include details about how to enable the index mode. Those (and other advanced) details are underway in elastic/elasticsearch#151739 If those steps ought to be moved, that can occur as needed.
Generative AI disclosure
Tool(s) and model(s) used: Cursor Grok 4.5