fix(db): add time_sensitive to oc_jobs index for maintenance window performance#58540
Open
turanalmammadov wants to merge 1 commit intonextcloud:masterfrom
Open
Conversation
…erformance
The background job scheduler uses this query to find eligible jobs:
SELECT * FROM oc_jobs
WHERE reserved_at <= ?
AND last_checked <= ?
AND time_sensitive = ?
ORDER BY last_checked ASC
LIMIT ?
The previous index 'job_lastcheck_reserved' only covered (last_checked,
reserved_at), forcing a partial index scan plus individual row reads to
evaluate the time_sensitive predicate. Under heavy load this causes
unnecessary I/O especially during a maintenance window where most rows
will be filtered out by the time_sensitive condition.
Changes:
1. New migration drops 'job_lastcheck_reserved' and creates a replacement
index 'job_lastcheck_timesens' on (last_checked, reserved_at, time_sensitive).
The new index is a covering index for the scheduler query, eliminating
the row-level time_sensitive evaluation entirely.
2. AddMissingIndicesListener updated to expect the new index name and columns
so that occ db:add-missing-indices picks it up on existing installations.
Fixes: nextcloud#46126
Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Hey @turanalmammadov, thanks for the pull request, and thanks for being transparent with your usage of AI. Can you double-check that this won't impact other request negatively? I don't know if such requests exist, but we might have some that do not include the time sensitive filter, and if so, I think those requests won't be able to use the index anymore. Also, can you share more regarding the actual impact in terms of numbers? How many rows do you have in your |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #46126
Problem
The background job scheduler uses this query to pick the next job:
The old index
job_lastcheck_reservedonly covered(last_checked, reserved_at). The database had to do a partial index scan and then fetch each row individually to evaluatetime_sensitive = ?. Under load with a maintenance window this causes many unnecessary I/O operations since most rows are excluded by thetime_sensitivecondition.Solution
Drop
job_lastcheck_reservedand replace it withjob_lastcheck_timesenson(last_checked, reserved_at, time_sensitive). Because the query's WHERE columns are now all in the index the database can use a covering index scan, evaluating thetime_sensitivepredicate without reading any data rows.Changes
1. New DB migration
core/Migrations/Version33000Date20260224000000.phpjob_lastcheck_reservedjob_lastcheck_timesenswith all three columns2. AddMissingIndicesListener update
core/Listener/AddMissingIndicesListener.phpocc db:add-missing-indicesrepairs existing installationsImpact
Made with Cursor