Add functional transcendentals + scalar-affine ops for scientific forward models#5
Open
NicolasRouquette wants to merge 1 commit into
Conversation
…ft/affine)
Surface differentiable elementwise exp/log and scalar-affine maps on the
nn.functional API, so scientific forward models (radiative transfer, dielectric
mixing, kinetics — built from exp/log of an affine argument rather than NN
activations) can be written as a pure autograd.fn1.Fn and differentiated by the
autograd engine, with no hand-coded gradient.
- F.{exp,log,scale,shift,affine} in Functional/Core.lean: thin lifts of the
existing Ops.{exp,log,scale,const} primitives, which already carry registered
backwards, so reverse-mode jacrev/grad works through them unchanged.
- Surfaced on nn.functional via both export bridges (FunctionalBatch + Seeded).
- Self-checking positive/negative example (NN/Examples/Functional/Transcendentals,
`lake exe transcendentals_check`): autograd gradients of exp, 3x+1, and exp(-2x)
match the closed form; a wrong-sign control is rejected.
- Blueprint chapter Ch2_Frontend/ScientificForwardModels.
Note: the example exe links the toolchain libc++ at runtime (native autograd
backend); set LD_LIBRARY_PATH to <toolchain>/lib accordingly.
3ee4e1e to
9179092
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A small, single-purpose contribution: surface differentiable elementwise
exp/logand scalar-affine maps on thenn.functionalAPI, so scientific forwardmodels — radiative transfer, dielectric mixing, kinetics, anything built from an
exp/logof an affine argument rather than from NN activations — can be writtenas a pure
autograd.fn1.Fnand differentiated by the autograd engine, with nohand-coded gradient.
Context
Independent and self-contained: this branch is based directly on
mainand doesnot depend on the factorization PRs (#2 and its eig/SVD follow-up). The diff is 7
files, +254 lines.
The motivating use case is a soil-moisture retrieval that combines SMAP (Soil
Moisture Active Passive) and NISAR (NASA–ISRO Synthetic Aperture Radar)
observations through the AVS (Attenuation–Volume–Surface) model, whose surface
term is
exp(-2·b·NDVI)·c·|R|². Operationally that model is fit per pixel with ahand-coded analytic Jacobian (plus a byte-duplicated JIT copy). A sign or factor
error there does not crash anything — it silently degrades the fit, and the copies
can drift. The ops here let the forward model be written once and its gradient be
derived by autograd, making the hand-coded Jacobian redundant.
What's here
NN/Runtime/Autograd/TorchLean/Functional/Core.lean):F.{exp, log, scale, shift, affine}— thin lifts of the existingOps.{exp, log, scale, const}primitives, which already carry registeredbackwards, so reverse-mode
jacrev/gradworks through them unchanged.affine x c k = c·x + k;scale/shiftare the multiply / add-by-constanthalves.
NN/API/Public/NN/FunctionalBatch.lean,NN/API/Public/Seeded.lean): the five ops added to bothnn.functionalexportbridges, beside
square/mean.NN/Examples/Functional/Transcendentals.lean,lake exe transcendentals_check): differentiatesexp,3x+1, andexp(-2x)and compares the autograd gradient to the closed form. Positive controls plus a
negative control — the autograd gradient of
exp(-2x)is-2·e^{-2x}and thetest rejects the wrong-sign
+2·e^{-2x}, i.e. exactly the hand-coded-Jacobiandefect class the ops are meant to eliminate. Exits non-zero on regression.
blueprint/.../Guide/Ch2_Frontend/ScientificForwardModels.lean):a short chapter documenting the ops and the autograd-derived-gradient point.
Notes for reviewers
libc++at runtime (native autograd backend);run with
LD_LIBRARY_PATH=<toolchain>/lib.Opsor the backends changes; this is purely additive on thefunctional surface.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com