Block arithmetic on restricted floats - #286
Draft
maleadt wants to merge 15 commits into
Draft
Conversation
Replace the closed `RestrictedFloat` union with method-based dispatch so package extensions can register their own restricted float types (FP8, FP4) alongside the built-in `TFloat32`. Mirrors cuTile Python's `NumericDTypeCategories.RestrictedFloat`, which covers tfloat32 and all the FP8/FP4 formats. The `reduce`/`scan` call sites run inside the compilation pipeline, whose world is frozen at `cuTile.__init__`, so they cannot see methods added by extensions loaded afterwards. Route them through `Base.invokelatest`, like `lookup_dtype!` already does for `julia_to_tile_dtype!`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`Tile{T}` arithmetic that dispatches straight to the float intrinsics
(`addf`/`subf`/`negf`/`mulf`/`divf`) bypasses the scalar broadcast path, so a
restricted float element type reached tileiras unchecked and failed the MLIR
verifier with "operand #0 must be tile of f16 or bf16 or f32 or f64 values".
That hit both `tf32_tile + tf32_tile` and `f8_tile + f8_tile`.
Guard the six affected methods with `check_arithmetic`, which folds away for
arithmetic floats and leaves an unconditional throw otherwise; `lower_throws!`
turns that into a collected compile-time diagnostic.
The message is a module-level constant rather than an interpolation of `op` and
`T`: kernel-side throws only report their text when it reconstructs to a
compile-time constant, and a run-time `string(op, ..., T, ...)` degrades to
"ArgumentError was thrown". The operator and element type are still named, by
`check_arithmetic`'s own frame in the diagnostic's stacktrace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DLFP8Types and Microfloats both define scalar arithmetic as a Float32
round-trip. Inside a kernel that combines with cuTile's `ftof` constructor
overlays, so `f8_tile .+ f8_tile` compiled silently into ftof → addf → ftof:
an implicit upcast with an extra rounding per operation and no diagnostic.
cuTile Python rejects the same expression with "has non-arithmetic dtype".
Register both packages' types as restricted floats, and shadow the upstream
arithmetic with erroring overlays in `cuTileMethodTable` so the broadcast path
reports the same error as the tile-level operators. Conversions, comparisons
and MMA are untouched, as is all host-side arithmetic.
The registration covers every `FP8` / `Microfloat` subtype rather than only the
ones with a Tile IR dtype: the overlays are written against those same abstract
supertypes to shadow exactly the methods upstream defines, and a subtype that
was not registered would fall through the guard and silently return `nothing`.
Unary negation is an exact sign-bit flip upstream and would have compiled, but
the tile-level `-(::Tile{T})` lowers to `negf` and is blocked, and having `-x`
and `(-).(x)` disagree on the same tile is worse than rejecting both.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both `fma_e4m3` kernels did `muladd.(ta, tb, tc)` on FP8 tiles, which is exactly the implicit-upcast pattern that is now rejected: broadcast `muladd` expands to the scalar `x * y + z`, whose operators are blocked. Round each input through FP8 and back, then compute in Float32. That is what the test was really checking — that representable values survive the round-trip — and the inputs keep products and sums exact, so the assertion is unchanged. Use `.*`/`.+` rather than `muladd.`: there is no elementwise `muladd` on Float32 tiles (tile `muladd` is the MMA path, and the scalar one lowers to the unmapped `muladd_float`). The FP8 kernels only reached it through Base's generic `muladd(x, y, z) = x * y + z` fallback for non-IEEE floats. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cover the four rejection paths per element type: broadcast, the direct tile operator, unary math through broadcast, and tile × scalar. FP8 adds broadcast `muladd` (which expands to `x * y + z`), and Microfloats adds a Float4_E2M1FN case to pin that the overlays dispatching on `Microfloat` really do cover every variant. Also pin the two things that must keep working: comparisons still lower to `ftof` + `cmpf`, and host-side FP8 arithmetic is unaffected by the overlays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Name the actual error and show the cast that fixes it, and note that comparisons on the FP8/FP4 types stay available. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The codegen-time checks in `emit_reduce!` and the scan intrinsic resolve `is_restricted_float` through `invokelatest` because the compilation pipeline runs in the world frozen at `__init__`, before the extensions load. Nothing exercised that with an extension type: a frozen-world call would return `false` for FP8 and fall through to an opaque failure in the reduce body instead of the early diagnostic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The docs promise that comparisons on restricted floats stay available, and they did for Microfloats — whose upstream definitions upcast through Float32 and ride the extension's `ftof` constructor overlays. DLFP8Types instead implements `<`, `==`, and `isless` at the bit level (`bitcast`, `_fpint`, with `isnan`/`iszero` guards), which does not compile in kernels: the guards bitcast broadcast scalars, tripping a tile shape mismatch in codegen. Shadow them with an exact Float32 upcast. f8 → f32 conversion is exact and injective (NaNs map to NaN), so IEEE comparison on the upcast values matches the bit-level ordering, including the NaN and ±0 cases, making `@consistent_overlay` appropriate. `<=` gets a direct overlay too: it has no upstream method, and comparing once beats Base's `Real` fallback composing it from `<` and `==`. `isless` still does not compile, but no longer for an FP8-specific reason: Base's `isless(::Float32, ::Float32)` itself fails under broadcast for any float tile (its `isnan` guard hits the same scalar-vs-tile mismatch), so the test pinning it is marked broken until that underlying issue is fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
After the restricted-arithmetic rewrite these kernels rounded each input through FP8 and then computed entirely in Float32, on inputs chosen to make that exact — reducing them to the round-trip coverage directly above plus ordinary Float32 arithmetic. The pattern they originally pinned (elementwise FP8 muladd through the upstream fallback) is now rejected by design and covered by the restricted-arithmetic testsets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The operator was unused beyond appearing in the diagnostic's stacktrace, where the frame right below — the guarded method itself — already names it. `check_arithmetic(T)` simplifies all six tile-level guards and every blocking overlay in the extensions. The message stays a module-level constant so kernel-side throw reconstruction keeps reporting it verbatim. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restricted-float scalars only ever come into existence inside cuTile's own broadcast/map machinery: kernels cannot load one any other way, and tile-level ops never consult scalar methods. So gate that single choke point instead of shadowing upstream scalar methods one by one — conversion, `ifelse` selection and comparisons pass through, everything else is rejected before dispatch. Comparisons upcast their restricted operands to Float32 and re-apply, which is exact and injective for every restricted format, and produces the same ftof/ftof/cmpf lowering as before. TFloat32 broadcast now reports the restricted-float error instead of falling through to Base's `no_op_err`, so all restricted types share one message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The per-extension blocklists enumerated the upstream method tables at a point in time: the moment DLFP8Types or Microfloats gains an `abs`, `rem` or `hypot`, the implicit-upcast hole silently reopens. The broadcast/map gate covers every present and future upstream method by construction, so ~35 shadowing overlays and the DLFP8Types comparison replacements can go. That also removes the `@consistent_overlay` for `<=`, which had no upstream method of the same signature — a pattern the macro's own docs caution against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Conversion in both directions and `ifelse` selection are the operations that stay available besides comparisons; a lambda that only casts is not, which is the part of the contract that is easiest to get wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Selection joins comparisons in the "stays available" list, and element-wise application of a custom function is now explicitly out — the gate rejects it even when the lambda only casts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`map` now delegates to `_apply_broadcast` rather than the other way around, so pointing at operations.jl for the mixed-type implementation reads backwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Attempt at fixing #240. I'm not particularly happy with the solution yet; in fact this branch contains two approaches: