Add note on N-2 index restore behavior and quiescence requirement #6702
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -172,6 +172,11 @@ | |
|
|
||
| ^2^ $$$footnote-2$$$ Supported with [searchable snapshots](/deploy-manage/tools/snapshot-and-restore/searchable-snapshots.md). | ||
|
|
||
| ::::{note} | ||
| :applies_to: stack: ga 9.2 | ||
| N-2 indices are restored as read-only. Snapshots taken during active indexing may fail to restore and could be missing recently indexed documents. If this happens, use a cluster running a version that fully supports the index, take a new snapshot there, and then restore that snapshot to the newer cluster. | ||
|
Member
There was a problem hiding this comment.
The term used elsewhere in these docs is "archive indices". I also think this info would be better-placed on https://www.elastic.co/docs/deploy-manage/upgrade/deployment-or-cluster/reading-indices-from-older-elasticsearch-versions.
Are you sure this is the case? It doesn't sound correct to me.
Bleurgh. I mean you're right, but this is a very ugly workaround.
Contributor
There was a problem hiding this comment. N-2 is something to clarify for sure, as it would be very difficult to understand what we mean. What about something like this and with a link to the doc mentioned by David?
(of course assuming my understanding of
Contributor
There was a problem hiding this comment.
The previous doesn't inspire much confidence.... What about something like this? When taking snapshots from older clusters (for example, 7.x), stop or minimize active indexing before creating the snapshot to help ensure all recently indexed documents are included when restoring to a newer cluster.
Member
There was a problem hiding this comment.
We already have the appropriate cross-link in
I don't think this works. "Minimize" isn't enough, the restore will fail if there is even a single indexing request in-flight at the time the snapshot is taken. But also this applies more generally to all snapshots, not just these N-2-restored ones, and still I do not think it is accurate because snapshots are guaranteed to contain all recently-indexed documents anyway.
Contributor
There was a problem hiding this comment. ^^ Oh, thanks for the info.... then we should probably highlight it as a known issue or something like that. What I don't like of the current paragraph is that feels vague. Your statement saying that this will fail if there's a single indexing request in-flight is much clearer and actionable for example.
Member
There was a problem hiding this comment. Actually I think it's worse than "a single indexing request in-flight", because what we really mean is that the global checkpoint matches the max sequence number, and even when that last indexing request finishes the global checkpoint will still lag until the follow-up sync action completes. I don't think users have a way to detect that this operation is still in flight.
Member
Author
There was a problem hiding this comment.
The term "archive indices" in the docs covers ES 5, 6, and 7, but the codebase distinguishes between the two. 6.x and below use the legacy archive path with BWC codecs, while 7.x indices follow a separate read-only restore path introduced in the linked PR. Using "archive indices" here would be misleading.
Confirmed, the possible mismatch is checkpoint sync lag, not missing documents.
Member
There was a problem hiding this comment. Ok let's spell out the exact version ranges to which this note applies. The phrase "N-2 indices" is not meaningful to users.
Contributor
There was a problem hiding this comment.
You are right, I saw it in the PR. But as the snapshot restore is mentioned in the reading indices from older versions doc, if this changes the way the data is restored compared to archive-functionality (when it comes from a 7.x and not from a 6.x or 5.x snapshot), we should probably include a note in that document too (just if that's relevant for the end user). Feel free to ping me in private any of you if you want some help in order to determine the best places to update the docs and share the information. If you explain to me the new functionality, and the existing caveats or situations to avoid I'd happily collaborate. The first part of this PR appears to say that
I don't get all the details, but the objective should be to document a procedure that is expected to work. So if But please ignore my comment if I'm not getting this right. |
||
| :::: | ||
|
|
||
| You can’t restore an index to an earlier version of {{es}}. For example, you can’t restore an index created in 8.18.0 to a cluster running 8.15.0. | ||
|
|
||
| #### Restoring incompatible indices | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this applies to all versions, not just ≥9.2? For instance elastic/elasticsearch#93389.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My plan is to backport the fix down to v9.2, so added the gating.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The 9.2 branch is now unmaintained (no more releases planned) so we shouldn't be backporting anything there. But also this note applies to 9.0 and 9.1 right? Snapshots will fail to restore in these versions as well.