[LAPACK][rocSOLVER] Fix incorrect QR results on repeated runs (#626) - #758
Open
zjin-lcf wants to merge 7 commits into
Open
[LAPACK][rocSOLVER] Fix incorrect QR results on repeated runs (#626)#758zjin-lcf wants to merge 7 commits into
zjin-lcf wants to merge 7 commits into
Conversation
The Householder-family cuSOLVER routines (geqrf, orgqr, ormqr, gebrd, orgbr, orgtr, ormtr and the complex ung*/unm* variants) passed nullptr for cuSOLVER's devInfo argument and skipped the lapack_info_check that every other routine (getrf, potrf, ...) performs. Besides losing all error reporting, this omitted the implicit synchronization that lapack_info_check performs (it reads devInfo back via a blocking queue.wait()). Without it, the SYCL event returned from the native-command submission could signal before the cuSOLVER kernels had finished, so a subsequent memcpy read partially-computed data. This produced nondeterministic, size-dependent wrong results - e.g. QR of a diagonal matrix returning Q diagonals stuck at the input value on the second and later runs for n >= 256 (issue uxlfoundation#626). Allocate a real devInfo and call lapack_info_check in all of these routines (buffer and USM paths), matching the established pattern. This both restores error checking and removes the race. Also align CusolverScopedContextHandler::get_stream with the cuBLAS backend by returning the interop handle's native queue (ih.get_native_queue()) instead of the queue's default stream, so cuSOLVER work is enqueued on the stream the SYCL runtime tracks for native-command completion. Fixes uxlfoundation#626.
Adds a functional test that repeatedly runs geqrf + orgqr on a known diagonal matrix (whose Q is the identity) and verifies Q's diagonal is 1 on every run. This guards against a regression of the cuSOLVER synchronization bug from uxlfoundation#626, where run 2+ returned corrupted results. Co-authored-by: Cursor <cursoragent@cursor.com>
…LVER A failed malloc_device silently degraded into passing a null devInfo to cuSOLVER, which disables the routine's error reporting - exactly the situation that hid the bug from uxlfoundation#626. Route every USM devInfo allocation through a create_devinfo helper that throws device_bad_alloc instead. Co-authored-by: Cursor <cursoragent@cursor.com>
…sion test The test creates a default, out-of-order queue, so the USM orgqr call was not ordered after geqrf. Pass the geqrf event as a dependency; the buffer path keeps relying on accessor ordering. The test needs no reference implementation, so stop dropping the whole LAPACK domain when Netlib LAPACKE is absent - only the tests that compare against a reference are skipped now. Co-authored-by: Cursor <cursoragent@cursor.com>
rocSOLVER calls only enqueue work on the HIP stream of the surrounding host task or native command. A submission that depends on that work may be scheduled on a different stream of the queue's stream pool without being ordered against this one, so it can read a factorization that has not finished yet. The Householder routines (geqrf, orgqr, ...) have no `info` output, so unlike getrf/potrf/... they do not get an implicit host-side wait from `lapack_info_check` that hides the problem. Wait for the stream before returning from the rocSOLVER call, which is what this backend already did on toolchains without `ext_codeplay_enqueue_native_command`. On an MI100 this turns the geqrf+orgqr diagonal regression test for uxlfoundation#626 from failing on repeated runs into passing. Co-authored-by: Cursor <cursoragent@cursor.com>
`ext_codeplay_enqueue_native_command` requires work to be enqueued on the stream of the interop handle, which is the stream the SYCL runtime records the submission's completion event on. Query it the same way the cuBLAS and cuSOLVER backends do instead of asking the queue for its native stream. Both currently resolve to the same stream inside a native command, so this is an alignment with the other backends rather than a behavioural change. Co-authored-by: Cursor <cursoragent@cursor.com>
…SOLVER A failed `malloc_device` for the `devInfo` output silently disabled the routine's error reporting, since rocSOLVER treats the resulting null pointer as "do not report". Throw `device_bad_alloc` instead, matching the cuSOLVER backend. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #748: the same #626 failure happens on the rocSOLVER backend, and this PR fixes it there.
Stacked on #748 — it is branched off that PR's head, so until #748 merges the diff here also contains its commits. The rocSOLVER-specific work is the last three commits.
The failure on ROCm
The
geqrf+orgqrdiagonal regression test added in #748 is backend agnostic, and it fails out of the box on an AMD Instinct MI100 (gfx908, ROCm 7.1.1), with exactly the #626 symptom — for a diagonal input the diagonal ofQstays at the input value on some runs:Root cause
rocSOLVER calls only enqueue work on the HIP stream of the surrounding host task / native command. When
ext_codeplay_enqueue_native_commandis available, this backend skips thehipStreamSynchronizeit otherwise performs, so the call returns with the factorization still in flight.That is only safe if a dependent submission is ordered against that stream, and it is not:
urEnqueueNativeCommandExppicks the stream throughgetNextComputeStream(NumEventsInWaitList, EventWaitList, Guard), which merely tries to reuse the stream of a dependency and enqueues nohipStreamWaitEventfor the wait list. When the heuristic declines,orgqrruns on a second stream that never waits forgeqrf.Unlike
getrf/potrf/..., the Householder routines have noinfooutput in rocSOLVER, so they also do not get the implicit host-side wait thatlapack_info_checkperforms — which is what makes the cuSOLVER fix in #748 work. There is no equivalent argument to pass here.Reduced to plain SYCL + rocSOLVER (no oneMath), on the MI100:
hipStreamWaitEventbetween themThe same reduction with plain
hipMemsetAsyncinstead of rocSOLVER does not fail, because a single stream is used often enough for the heuristic to hold.Changes
ext_codeplay_enqueue_native_command; the unsynchronized path is dropped rather than kept for a case where it is not sound. Fixing the ordering in the UR HIP adapter (both it and the CUDA one look affected) would let this be relaxed again.devInfoallocation, mirroring thecreate_devinfohelper [LAPACK][cuSOLVER] Fix incorrect QR results on repeated runs (#626) #748 adds to cuSOLVER.Test plan — AMD Instinct MI100, ROCm 7.1.1, oneMath built with
-DENABLE_ROCSOLVER_BACKEND=ON -DTARGET_DOMAINS=lapackand an open-source DPC++ (intel/llvm, HIP backend)geqrf+orgqrregression test from #748, n = 256 / 512 / 1024, 6 runs each, buffer and USM, RT and CT APIs:Full LAPACK suite against Netlib reference (ILP64 LAPACKE), run one suite per process, unpatched vs. patched:
172 passed with this PR vs. 169 unpatched; the difference is exactly the QR regression tests.
112 failures remain, all in the
*BatchGroupsuites, which throwunimplementedfor the group-batch APIs. Identical on the unpatched build.Getrf/GetrsUSM abort in both builds inside the DPC++ runtime (ProgramManager::getDeviceKernelInfoassertion) before reaching any oneMath code; unrelated to this change.Reproduced [cuSOLVER] Incorrect results in QR decomposition on CUDA #626 on ROCm and confirmed the fix.
No change in the rest of the rocSOLVER LAPACK suite.
CI: rocSOLVER LAPACK functional tests.
The rocBLAS, cuBLAS and cuSOLVER backends share the same
#ifdef SYCL_EXT_ONEAPI_ENQUEUE_NATIVE_COMMANDpattern and are exposed to the same hazard; I have left them alone here.