Add note on N-2 index restore behavior and quiescence requirement#6702
Add note on N-2 index restore behavior and quiescence requirement#6702reugn wants to merge 1 commit into
Conversation
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
🔍 Preview links for changed docs |
Vale Linting ResultsSummary: 1 suggestion found 💡 Suggestions (1)
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
| ^2^ $$$footnote-2$$$ Supported with [searchable snapshots](/deploy-manage/tools/snapshot-and-restore/searchable-snapshots.md). | ||
|
|
||
| ::::{note} | ||
| :applies_to: stack: ga 9.2 |
There was a problem hiding this comment.
I think this applies to all versions, not just ≥9.2? For instance elastic/elasticsearch#93389.
There was a problem hiding this comment.
My plan is to backport the fix down to v9.2, so added the gating.
There was a problem hiding this comment.
The 9.2 branch is now unmaintained (no more releases planned) so we shouldn't be backporting anything there. But also this note applies to 9.0 and 9.1 right? Snapshots will fail to restore in these versions as well.
|
|
||
| ::::{note} | ||
| :applies_to: stack: ga 9.2 | ||
| N-2 indices are restored as read-only. Snapshots taken during active indexing may fail to restore and could be missing recently indexed documents. If this happens, use a cluster running a version that fully supports the index, take a new snapshot there, and then restore that snapshot to the newer cluster. |
There was a problem hiding this comment.
N-2 indices
The term used elsewhere in these docs is "archive indices". I also think this info would be better-placed on https://www.elastic.co/docs/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.
could be missing recently indexed documents
Are you sure this is the case? It doesn't sound correct to me.
use a cluster running a version that fully supports the index
Bleurgh. I mean you're right, but this is a very ugly workaround.
There was a problem hiding this comment.
N-2 is something to clarify for sure, as it would be very difficult to understand what we mean.
What about something like this and with a link to the doc mentioned by David?
Indices created two major versions earlier than the current {{es}} version (N-2 indices) are automatically restored as read-only. Refer to Reading indices from older {{es}} versions for more information.
(of course assuming my understanding of N-2 indices is accurate :) )
There was a problem hiding this comment.
Snapshots taken during active indexing may fail to restore and could be missing recently indexed documents. If this happens, use a cluster running a version that fully supports the index, take a new snapshot there, and then restore that snapshot to the newer cluster.
The previous doesn't inspire much confidence.... What about something like this?
When taking snapshots from older clusters (for example, 7.x), stop or minimize active indexing before creating the snapshot to help ensure all recently indexed documents are included when restoring to a newer cluster.
There was a problem hiding this comment.
What about something like this and with a link to the doc mentioned by David?
We already have the appropriate cross-link in $$$footnote-1$$$ above, I don't think we need another one on this page.
When taking snapshots from older clusters (for example, 7.x), stop or minimize active indexing before creating the snapshot to help ensure all recently indexed documents are included when restoring to a newer cluster.
I don't think this works. "Minimize" isn't enough, the restore will fail if there is even a single indexing request in-flight at the time the snapshot is taken. But also this applies more generally to all snapshots, not just these N-2-restored ones, and still I do not think it is accurate because snapshots are guaranteed to contain all recently-indexed documents anyway.
There was a problem hiding this comment.
^^ Oh, thanks for the info.... then we should probably highlight it as a known issue or something like that.
What I don't like of the current paragraph is that feels vague. Your statement saying that this will fail if there's a single indexing request in-flight is much clearer and actionable for example.
There was a problem hiding this comment.
Actually I think it's worse than "a single indexing request in-flight", because what we really mean is that the global checkpoint matches the max sequence number, and even when that last indexing request finishes the global checkpoint will still lag until the follow-up sync action completes. I don't think users have a way to detect that this operation is still in flight.
There was a problem hiding this comment.
The term used elsewhere in these docs is "archive indices".
The term "archive indices" in the docs covers ES 5, 6, and 7, but the codebase distinguishes between the two. 6.x and below use the legacy archive path with BWC codecs, while 7.x indices follow a separate read-only restore path introduced in the linked PR. Using "archive indices" here would be misleading.
Are you sure this is the case?
Confirmed, the possible mismatch is checkpoint sync lag, not missing documents.
There was a problem hiding this comment.
Ok let's spell out the exact version ranges to which this note applies. The phrase "N-2 indices" is not meaningful to users.
There was a problem hiding this comment.
The term "archive indices" in the docs covers ES 5, 6, and 7, but the codebase distinguishes between the two. 6.x and below use the legacy archive path with BWC codecs, while 7.x indices follow a separate read-only restore path introduced in the linked PR. Using "archive indices" here would be misleading.
You are right, I saw it in the PR.
But as the snapshot restore is mentioned in the reading indices from older versions doc, if this changes the way the data is restored compared to archive-functionality (when it comes from a 7.x and not from a 6.x or 5.x snapshot), we should probably include a note in that document too (just if that's relevant for the end user).
Feel free to ping me in private any of you if you want some help in order to determine the best places to update the docs and share the information. If you explain to me the new functionality, and the existing caveats or situations to avoid I'd happily collaborate.
The first part of this PR appears to say that Indices created two major versions earlier than the current {{es}} version (N-2 indices) are automatically restored as read-only.. IMO that's a good addition.
Actually I think it's worse than "a single indexing request in-flight"
I don't get all the details, but the objective should be to document a procedure that is expected to work. So if Snapshots taken during active indexing may fail to restore is likely to happen the procedure should require to stop all indexing before taking a snapshot for this functionality (restore in a major+2 cluster) to work. Doing that we should probably be on the safe side.
But please ignore my comment if I'm not getting this right.
3 participants
Related to elastic/elasticsearch#145214, which adds support for restoring N-2 indices as read-only.
Adds a note to the index compatibility section explaining: