Skip to content

Fix line number handling in @kernel - #734

Open
vchuravy wants to merge 2 commits into
mainfrom
vc/kernel-linenumbers
Open

Fix line number handling in @kernel#734
vchuravy wants to merge 2 commits into
mainfrom
vc/kernel-linenumbers

Conversation

@vchuravy

@vchuravy vchuravy commented Jul 27, 2026

Copy link
Copy Markdown
Member

Fixes #732.

@kernel spliced LineNumberNodes pointing into KernelAbstractions' own sources into the generated code, and dropped some of the user's line information entirely. GPUCompiler emits device coverage by walking the debug info of :code_coverage_effect statements at compile time (record_coverage in jlgen.jl), so bad line info directly becomes bad coverage: kernel lines get reported as untracked, and KernelAbstractions' own sources get credited instead.

Reproduction

kernels.jl, run under julia --code-coverage, against CPU() (POCL):

@kernel function mul2!(a)
    i = @index(Global)
    x = a[i]
    a[i] = 2 * x
end

@kernel function sync2!(a)
    i = @index(Local)
    lm = @localmem Float64 (8,)
    lm[i] = a[i]
    @synchronize
    a[i] = lm[i] + 1
end

Before — five of the kernels' lines are reported as untracked (-) even though both kernels were compiled and run:

        - @kernel function mul2!(a)
        1     i = @index(Global)
        1     x = a[i]
        1     a[i] = 2 * x
        - end
        -
        - @kernel function sync2!(a)
        1     i = @index(Local)
        -     lm = @localmem Float64 (8,)
        1     lm[i] = a[i]
        -     @synchronize
        -     a[i] = lm[i] + 1
        - end

After — every line is accounted for:

        2 @kernel function mul2!(a)
        1     i = @index(Global)
        1     x = a[i]
        1     a[i] = 2 * x
        - end
        -
        2 @kernel function sync2!(a)
        1     i = @index(Local)
        1     lm = @localmem Float64 (8,)
        1     lm[i] = a[i]
        1     @synchronize
        1     a[i] = lm[i] + 1
        - end

The lost coverage did not vanish, it was charged to src/macros.jl. From the same pre-fix run:

        3 function emit(loop)
        3     stmts = Any[]
        -
        3     body = Expr(:block, loop.stmts...)
        3     loopexpr = quote
        -         $(loop.allocations...)
        1         if __active_lane__
        2             $(unblock(body))

Causes

  • emit built the workitem loop with a quote block, so every emitted block started with a macros.jl line node. It now builds the expression directly.
  • MacroTools.unblock strips line nodes when it collapses a single-statement block. This silently discarded the line of the first statement after a @synchronize — that statement ended up with no line information at all, which is why a[i] = lm[i] + 1 above reads as untracked. Replaced with a variant that only unwraps blocks carrying no line info.
  • split hoists @uniform / @localmem / @private out of the workitem loop but left their line nodes behind in the loop body. The pending line node is now tracked and moved along with the statement.

Additionally:

  • The generated constructor functions are relocated to the @kernel call site, so the @kernel function … line is attributed to the user's file instead of macros.jl:39-46. That is the -2 change on the @kernel lines above.
  • @groupsize, @ndrange, @localmem, @private, @synchronize and @print return bare expressions instead of quote blocks, so they no longer inject KernelAbstractions.jl line nodes into kernel bodies.

Testing

Two new test files, both run from runtests.jl only (they are backend-independent, so there is no reason to make every backend package pay for them):

  • test/linenumbers.jl asserts that a @kernel expansion only ever refers to the file it was written in, and that every source line of the kernel is represented. Covers the plain case, @synchronize plus hoisted allocations, @uniform/@private, the inbounds=/unsafe_indices= configurations, and the kernel-language macros.
  • test/coverage.jl is the end-to-end check: it runs the script above in a subprocess under --code-coverage and asserts every kernel line is tracked in the resulting LCOV tracefile. Against the pre-fix expansion it fails on exactly the five lines shown above. Takes ~60-75s, since coverage forces the subprocess to recompile rather than use cached native code.

Coverage is written to an LCOV tracefile in a temp dir rather than via --code-coverage=user, which would drop a .cov file next to every tracked source in both the checkout and the depot, and would perturb the outer report when the suite itself runs under Pkg.test(coverage=true).

Full suite passes (3125 pass, 4 pre-existing broken), Runic clean.

Known limitation, not addressed here

Device coverage is only recorded for --code-coverage=user and --code-coverage=all. The --code-coverage=@<path> mode records nothing for device code — verified empirically, and it is the mode itself rather than which files are tracked (pointing @<path> at a tree containing both the script and KernelAbstractions' sources still records nothing).

That matters because Pkg.test(coverage=true) passes --code-coverage=@$(pkgroot), which is what the standard julia-actions/julia-runtest CI setup uses. So under a typical Codecov setup this PR upgrades the @kernel function … signature line from untracked to tracked, but kernel body lines will still be missing. That gap is in GPUCompiler/Julia, not here.

Device coverage also means "this code was compiled", with counts reflecting compilations rather than executions — inherent to GPUCompiler's approach, since device code cannot call into the Julia runtime.

🤖 Generated with Claude Code

The `@kernel` expansion spliced `LineNumberNode`s pointing into
KernelAbstractions' own sources into the generated code, and dropped some
of the user's line information entirely. Coverage and profiling tools then
attributed kernel bodies to `src/macros.jl` instead of the user's file.

Three separate causes:

- `emit` built the workitem loop with a `quote` block, so every emitted
  block started with a `macros.jl` line node. Build the expression
  directly instead.
- `MacroTools.unblock` strips line nodes when collapsing a single-statement
  block, which silently discarded the line of the first statement after a
  `@synchronize`. Use a variant that only unwraps blocks without line
  information.
- `split` hoists `@uniform`/`@localmem`/`@private` out of the workitem
  loop, leaving their line nodes behind in the loop body. Track the pending
  line node and move it along with the statement.

The generated constructor functions are now attributed to the `@kernel`
call site rather than to `macros.jl`, and the kernel-language macros
(`@groupsize`, `@ndrange`, `@localmem`, `@private`, `@synchronize`,
`@print`) return bare expressions so they no longer inject
`KernelAbstractions.jl` line nodes into kernel bodies.

Fixes #732

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main 222425e... main / 222425e...
saxpy/default/Float32/1024 0.0577 ± 0.036 ms 0.0446 ± 0.036 ms 1.29 ± 1.3
saxpy/default/Float32/1048576 0.509 ± 0.061 ms 0.513 ± 0.036 ms 0.991 ± 0.14
saxpy/default/Float32/16384 0.0531 ± 0.029 ms 0.0521 ± 0.028 ms 1.02 ± 0.78
saxpy/default/Float32/2048 0.0633 ± 0.029 ms 0.061 ± 0.03 ms 1.04 ± 0.69
saxpy/default/Float32/256 0.0582 ± 0.035 ms 0.0491 ± 0.036 ms 1.19 ± 1.1
saxpy/default/Float32/262144 0.18 ± 0.026 ms 0.175 ± 0.026 ms 1.03 ± 0.21
saxpy/default/Float32/32768 0.0599 ± 0.029 ms 0.0589 ± 0.029 ms 1.02 ± 0.71
saxpy/default/Float32/4096 0.0675 ± 0.028 ms 0.0664 ± 0.027 ms 1.02 ± 0.59
saxpy/default/Float32/512 0.0546 ± 0.036 ms 0.0447 ± 0.036 ms 1.22 ± 1.3
saxpy/default/Float32/64 0.0511 ± 0.036 ms 0.0486 ± 0.036 ms 1.05 ± 1.1
saxpy/default/Float32/65536 0.0796 ± 0.031 ms 0.0764 ± 0.031 ms 1.04 ± 0.58
saxpy/default/Float64/1024 0.0604 ± 0.033 ms 0.06 ± 0.032 ms 1.01 ± 0.77
saxpy/default/Float64/1048576 0.539 ± 0.14 ms 0.554 ± 0.12 ms 0.972 ± 0.33
saxpy/default/Float64/16384 0.0557 ± 0.03 ms 0.0561 ± 0.03 ms 0.993 ± 0.75
saxpy/default/Float64/2048 0.0626 ± 0.027 ms 0.0635 ± 0.028 ms 0.986 ± 0.62
saxpy/default/Float64/256 0.0458 ± 0.036 ms 0.0549 ± 0.036 ms 0.835 ± 0.86
saxpy/default/Float64/262144 0.182 ± 0.018 ms 0.181 ± 0.019 ms 1.01 ± 0.15
saxpy/default/Float64/32768 0.0653 ± 0.031 ms 0.0645 ± 0.031 ms 1.01 ± 0.68
saxpy/default/Float64/4096 0.0667 ± 0.027 ms 0.0653 ± 0.027 ms 1.02 ± 0.59
saxpy/default/Float64/512 0.0641 ± 0.036 ms 0.0475 ± 0.036 ms 1.35 ± 1.3
saxpy/default/Float64/64 0.0554 ± 0.036 ms 0.0486 ± 0.036 ms 1.14 ± 1.1
saxpy/default/Float64/65536 0.0875 ± 0.028 ms 0.0828 ± 0.028 ms 1.06 ± 0.5
saxpy/static workgroup=(1024,)/Float32/1024 0.0645 ± 0.033 ms 0.0453 ± 0.036 ms 1.42 ± 1.3
saxpy/static workgroup=(1024,)/Float32/1048576 0.458 ± 0.024 ms 0.451 ± 0.029 ms 1.01 ± 0.083
saxpy/static workgroup=(1024,)/Float32/16384 0.0505 ± 0.028 ms 0.0499 ± 0.027 ms 1.01 ± 0.78
saxpy/static workgroup=(1024,)/Float32/2048 0.062 ± 0.026 ms 0.0604 ± 0.03 ms 1.03 ± 0.66
saxpy/static workgroup=(1024,)/Float32/256 0.0592 ± 0.036 ms 0.0482 ± 0.036 ms 1.23 ± 1.2
saxpy/static workgroup=(1024,)/Float32/262144 0.163 ± 0.03 ms 0.161 ± 0.03 ms 1.01 ± 0.26
saxpy/static workgroup=(1024,)/Float32/32768 0.0556 ± 0.028 ms 0.0551 ± 0.028 ms 1.01 ± 0.72
saxpy/static workgroup=(1024,)/Float32/4096 0.0655 ± 0.025 ms 0.0649 ± 0.026 ms 1.01 ± 0.56
saxpy/static workgroup=(1024,)/Float32/512 0.0564 ± 0.036 ms 0.038 ± 0.036 ms 1.48 ± 1.7
saxpy/static workgroup=(1024,)/Float32/64 0.0556 ± 0.036 ms 0.0399 ± 0.036 ms 1.39 ± 1.5
saxpy/static workgroup=(1024,)/Float32/65536 0.0709 ± 0.03 ms 0.0698 ± 0.03 ms 1.02 ± 0.61
saxpy/static workgroup=(1024,)/Float64/1024 0.0593 ± 0.032 ms 0.0563 ± 0.034 ms 1.05 ± 0.86
saxpy/static workgroup=(1024,)/Float64/1048576 0.479 ± 0.017 ms 0.471 ± 0.021 ms 1.02 ± 0.059
saxpy/static workgroup=(1024,)/Float64/16384 0.053 ± 0.028 ms 0.0527 ± 0.028 ms 1 ± 0.75
saxpy/static workgroup=(1024,)/Float64/2048 0.0607 ± 0.027 ms 0.0621 ± 0.028 ms 0.978 ± 0.62
saxpy/static workgroup=(1024,)/Float64/256 0.055 ± 0.036 ms 0.0374 ± 0.036 ms 1.47 ± 1.7
saxpy/static workgroup=(1024,)/Float64/262144 0.172 ± 0.028 ms 0.169 ± 0.029 ms 1.02 ± 0.24
saxpy/static workgroup=(1024,)/Float64/32768 0.0614 ± 0.03 ms 0.059 ± 0.03 ms 1.04 ± 0.73
saxpy/static workgroup=(1024,)/Float64/4096 0.0643 ± 0.026 ms 0.0635 ± 0.026 ms 1.01 ± 0.58
saxpy/static workgroup=(1024,)/Float64/512 0.0496 ± 0.036 ms 0.0494 ± 0.036 ms 1 ± 1
saxpy/static workgroup=(1024,)/Float64/64 0.0524 ± 0.036 ms 0.0469 ± 0.036 ms 1.12 ± 1.2
saxpy/static workgroup=(1024,)/Float64/65536 0.0831 ± 0.03 ms 0.0768 ± 0.03 ms 1.08 ± 0.58
time_to_load 0.932 ± 0.0058 s 0.946 ± 0.016 s 0.985 ± 0.017

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.75%. Comparing base (65690fd) to head (222425e).

Files with missing lines Patch % Lines
src/macros.jl 0.00% 35 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (65690fd) and HEAD (222425e). Click for more details.

HEAD has 24 uploads less than BASE
Flag BASE (65690fd) HEAD (222425e)
52 28
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #734       +/-   ##
==========================================
- Coverage   64.77%   0.75%   -64.02%     
==========================================
  Files          24      23        -1     
  Lines        2024    1853      -171     
==========================================
- Hits         1311      14     -1297     
- Misses        713    1839     +1126     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Runs a script defining two kernels in a subprocess under `--code-coverage`,
then asserts that every line of both kernel bodies is reported as tracked in
the resulting LCOV tracefile.

GPUCompiler records device coverage by visiting the source location of each
`:code_coverage_effect` while compiling (`record_coverage` in jlgen.jl), so
this exercises the actual consumer of the line information that `@kernel`
emits. Against the previous macro expansion it fails on five lines: both
`@kernel function` signatures, the hoisted `@localmem`, the `@synchronize`
and the statement following it.

Coverage is written to an LCOV tracefile inside a temporary directory rather
than using `--code-coverage=user`, which would drop a `.cov` file next to
every tracked source file in both the checkout and the depot, and would
perturb the outer report when the suite runs under `Pkg.test(coverage=true)`.

Note the `=@path` mode that `Pkg.test(coverage=true)` selects does not record
device coverage at all, independently of this change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vchuravy
vchuravy marked this pull request as ready for review July 27, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Code coverage tracking of kernels does not work correctly

1 participant