feat(supertask): skip pretasks already completed against the hashlist#2177
Open
linuxkd wants to merge 1 commit into
Open
feat(supertask): skip pretasks already completed against the hashlist#2177linuxkd wants to merge 1 commit into
linuxkd wants to merge 1 commit into
Conversation
Add an opt-in option to skip pretasks whose equivalent attack has already been fully exhausted against the target hashlist when applying a supertask, avoiding duplicate tasks that re-run identical work for zero new coverage. Default off preserves existing behavior for all callers. - TaskUtils::findCompletedEquivalent matches a pretask spec to a fully exhausted Task on the hashlist (normalized attackCmd, fileId set, cracker binary+type, keyspaceProgress >= keyspace > 0; archived counts) - SupertaskUtils::runSupertask gains $skipCompleted; returns the created TaskWrapper plus the list of skipped pretasks, and creates no wrapper when every pretask is skipped - apiv2 createSupertask gains a skipCompleted flag and reports skipped pretasks via the top-level JSON:API meta member (TaskWrapper response unchanged when the flag is off) - legacy supertask-apply forms get a "skip completed" checkbox - phpunit unit and e2e tests covering the match, every exclusion, and the runSupertask skip flow Refs hashtopolis#2167
Author
|
Web-ui follow-up PR (the Angular checkbox referenced above): hashtopolis/web-ui#688 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the opt-in dedup requested in #2167. When applying a supertask to a hashlist, optionally skip any pretask whose equivalent attack has already been fully exhausted against that hashlist, instead of re-instantiating a duplicate task that re-runs identical work for zero new coverage.
Opt-in, default OFF — when
skipCompletedis not set, behavior is identical to today for every caller (legacy UI, apiv2, and internalrunSupertaskcallers).Motivation
It's common to apply a refined supertask after an earlier one has finished. Without dedup, every overlapping pretask spawns a fresh task that re-runs an attack already exhausted against the same hashlist. In one recent case, 9 of 40 pretasks in the second supertask were exact duplicates of completed tasks — pure wasted compute.
Match criteria
A Task
Tis an already-completed equivalent of a pretask spec on hashlistHwhen ALL hold:attackCmdequality (whitespace collapsed/trimmed; flag order not normalized — deliberately conservative)Tvia FileTask vs pretask via FilePretask)T.crackerBinaryId== resolved binary id ANDT.crackerBinaryTypeId== type idT.keyspace > 0ANDT.keyspaceProgress >= T.keyspace(fully exhausted; partials never match)Tbelongs to a TaskWrapper whosehashlistId == HArchived-but-exhausted tasks do count as matches.
API response shape
apiv2
helper/createSupertaskgains an optionalskipCompletedbool. When set, the response adds a top-level JSON:APImeta.skippedPretasksarray alongside the unchangedTaskWrapperdataresource:{ "data": { "...": "TaskWrapper" }, "meta": { "skippedPretasks": [ { "pretaskId": 12, "matchingTaskId": 88 } ] } }When every pretask is skipped, no TaskWrapper is created and a meta-only response is returned:
{ "taskWrapperId": null, "skippedPretasks": [ "..." ] }Fully backward-compatible: the
metamember is only added whenskipCompletedis requested,getResponse()stays"TaskWrapper", and the new meta member is documented in theactionPostPHPDoc (feeds the OpenAPI description).What's in this PR
TaskUtils::findCompletedEquivalent()(+normalizeAttackCmd,getFileIdsOfTask,getFileIdsOfPretask)SupertaskUtils::runSupertask()gains$skipCompleted; returns['taskWrapper' => …, 'skippedPretasks' => …]; suppresses the empty wrapper when all are skippedgetOneResource($extraMeta)+ overridablegetExtraMeta(), threaded throughprocessPost(additive, non-breaking)helper/CreateSupertaskHelperAPI:skipCompletedfield + meta wiringfindCompletedEquivalent(match + every exclusion + whitespace normalization + archived), 3 e2e tests forrunSupertaskskip behaviorTesting
src/inc/apiv2+ci/phpunitci/apiv2/test_supertask.pypassesFollow-ups (intentionally out of scope here)
hashtopolis/web-ui(ready, opened right after this)findCompletedEquivalentis reusable and ready to power it laterRefs #2167