Skip to content

Merge functions and procedures in Laurel#28

Draft
keyboardDrummer wants to merge 329 commits into
mainfrom
issue-924-contract-and-proof-pass
Draft

Merge functions and procedures in Laurel#28
keyboardDrummer wants to merge 329 commits into
mainfrom
issue-924-contract-and-proof-pass

Conversation

@keyboardDrummer
Copy link
Copy Markdown
Owner

@keyboardDrummer keyboardDrummer commented Apr 23, 2026

Builds on:

Functional changes

  1. Merge functions and procedures:
    1. Allow postconditions on functions
    2. Allow calling procedures from contracts
    3. Allow transparent procedures IF they have an expressionish body
    4. Allow assertions and assumptions in functions
  2. Improve diagnostics related to contracts, using the correct verbiage "precondition" and "postcondition" instead of "assertion"

Implementation

Add these passes:

  • [New] EliminateReturnStatements: rewrite return to exit statements, needed for the next pass.
  • [New] ContractPass: translate away pre and postconditions entirely by introducing assertion and assumptions at call sites and at procedure starts and ends
  • [New] TransparencyPass: for each Core procedure, generate a function with the same signature and name suffixed with $asFunction. If a Core procedure is marked as transparent, attempt to generate a functional version of it, where assertions are erased and all calls are to functional versions. Tie the functional version to the procedure using a free postcondition.
  • [Updated] Lift assertions, assumptions and procedure calls when they occur in expressions.

Why can't we have a PR with just the contract or just the transparency pass?

  • Just the contract pass means that the preconditions which are currently supported on functions are no longer supported, because assertions and assumptions in functions are not supported.
  • Just the transparency pass means that making calls from contracts is no longer supported, since in the procedure world, all calls are to procedures and we can't lift procedure calls in Core contracts

Follow-up work

  • Remove the 'functional' keyword from the Laurel grammar
  • Lift assumptions in expressions to axioms.
  • In the transparency phase, if something has no asserts and only calls functions, only create a function and no procedure
  • Move the transparency phase so its Core->Core instead of UnorderedCoreWithLaurelTypes -> UnorderedCoreWithLaurelTypes

keyboardDrummer and others added 30 commits April 20, 2026 15:50
…opaqueSpec

- Changed procedureToOp to produce opaqueSpec op with ensures and modifies
  as nested args (matching the grammar), instead of separate top-level args
- Added missing opaque keyword in MapStmtExprTest and T2_ModifiesClauses tests
…lator

- Add `axioms : List StmtExprMd` field to Laurel's Procedure structure
- In ContractPass, build axiom expressions from invokeOn trigger + ensures
  clauses (nested Forall with trigger on innermost quantifier), then clear
  the invokeOn field
- Update LaurelToCoreTranslator to translate proc.axioms directly instead
  of using the now-removed translateInvokeOnAxiom function
- Update CoreGroupingAndOrdering to use axioms instead of invokeOn for
  procedure ordering and callee collection
- Update Resolution to preserve axioms through name resolution
- Update MapStmtExpr to traverse axioms in mapProcedureM
- Fix InvokeOn test expected error messages
- Grammar: Add `modifiesWildcard` op for `modifies *` syntax
- Parser: Handle `modifiesWildcard` by producing `StmtExpr.All` in modifies list
- Printer: Emit `modifiesWildcard` when modifies list contains `All`
- ModifiesClauses: Skip frame condition generation for wildcard modifies
- FilterNonCompositeModifies: Preserve `All` entries (don't filter as non-composite)
- HeapParameterization: Don't treat wildcard `All` as evidence of heap access
- PythonToLaurel: Use `modifies *` for all opaque procedures
- Specs/ToLaurel: Use `modifies *` for spec procedures
- T2_ModifiesClauses: Uncomment wildcard test case
Two issues were causing the Python test failures:

1. dbg_trace statements in LaurelCompilationPipeline.lean were printing
   massive debug output that polluted #guard_msgs test expectations.

2. The contract pass (ContractPass.lean) created Assert/Assume nodes
   with source := none (via mkMd). When these reached the Core
   translator, getNameFromMd triggered 'BUG: metadata without a
   filerange' debug traces, further polluting test output.

Fix: propagate source locations from the original preconditions and
body to the Assert/Assume/Block nodes created by transformProcBody.
Prefix `program` parameter with underscore to suppress the
'unused variable' warning that causes the docs build to fail
with --wfail.
Formatting improvements:
- Change block formatting to use indent(2) with newlines for vertical layout
  instead of single-line '{ ... }' format
- Update semicolon separator to use newlines instead of spaces

Debugging improvements:
- Replace boolean 'coreProgramHasSuperfluousErrors' with 'coreDiagnostics' list
  that records why the Core program was suppressed, enabling better error surfacing
- Add source location and reason parameters to 'invalidCoreType' for more
  informative diagnostics
- Surface core diagnostics when no other diagnostics explain program suppression
- Add 'processLaurelFileKeepIntermediates' test helper for writing intermediate
  pipeline files to Build/ directory
- Add Build/ to .gitignore

Test updates:
- Update all expected outputs to match new block formatting
- Add 'opaque' keyword to test procedures that need it for the new formatting
- Fix #guard_msgs whitespace formatting
Diagnostics with FileRange.unknown are not actionable for users and
can arise from pre-existing resolution limitations (e.g., variables
in multi-assign with field targets losing their uniqueId). Filter
these out when surfacing suppression reasons to avoid spurious errors.
The CodeBuild source location was hardcoded to strata-org/Strata.git,
which fails when the commit only exists in keyboardDrummer/Strata.
Use github.server_url/github.repository so it resolves to the correct
repo dynamically.
The CodeBuild project has credentials for strata-org/Strata only.
On forks like keyboardDrummer/Strata, the job cannot access the source.
Add an 'if' condition to skip the benchmark on non-upstream repos,
and restore the hardcoded source URL that CodeBuild expects.
…gging-improvements

# Conflicts:
#	Strata/Languages/Laurel/LaurelToCoreTranslator.lean
…t emitDiagnostic

- Remove the FileRange.unknown filter in LaurelCompilationPipeline: all
  coreDiagnostics are now surfaced when no other diagnostics explain
  the suppression.
- Remove default parameters from invalidCoreType, making source and
  reason explicit at every call site.
- Remove redundant emitDiagnostic in the catch-all arm of translateType:
  invalidCoreType already records the diagnostic via coreDiagnostics.
Diagnostics without a known source location are not actionable for the
user. Restore the filter that was removed in 0b6cee4 to fix the
T1_MutableFields test failure.
@keyboardDrummer
Copy link
Copy Markdown
Owner Author

@keyboardDrummer-bot can you create another PR that is a copy of this PR, but without the first two phases:
[New] EliminateReturnStatements: rewrite return to exit statements, needed for the next pass.
[New] ContractPass: translate away pre and postconditions entirely by introducing assertion and assumptions at call sites and at procedure starts and ends

You might run into issues with assertions not being lifted in the lifting pass.
See if there are fixes in #43, strata-org#1113 and strata-org#1130 that you can use. Actually, start with a merge of all the PRs related to those links.

@keyboardDrummer-bot
Copy link
Copy Markdown
Collaborator

I've created PR #45 as requested — it's a copy of this PR without the EliminateReturnStatements and ContractPass phases.

The PR includes the fixes from strata-org#1113 (HeapParam scope bug) and strata-org#1130 (lift imperative call prepend ordering) since those were already part of this PR's changes.

Key changes from this PR:

  • Removed ContractPass.lean and EliminateReturnStatements.lean
  • Removed their invocations from the pipeline
  • Restored invokeOn axiom handling in the translator
  • Reverted test expectations that depended on contract pass behavior (precondition error messages at call sites)

Some tests may need adjustment since without the contract pass:

  • Postconditions aren't assumed at call sites (callers can't prove properties about return values)
  • Precondition violations at call sites report "precondition does not hold" instead of "assertion could not be proved"
  • The T2_ImpureExpressionsError test may need updates depending on whether procedure calls in preconditions are still caught

@keyboardDrummer
Copy link
Copy Markdown
Owner Author

@keyboardDrummer-bot please update the contract pass in this PR so it mimics the design of #34. In particular:

Contract Pass Design
Key differences from the version in https://github.com/keyboardDrummer/Strata/pull/28:

$post procedures are functional (isFunctional := true)
$post takes all inputs AND all outputs of the original procedure as parameters
$post does not call the original procedure — it simply returns the conjunction of postconditions over the combined input+output parameters
At call sites, before the call, all input arguments are assigned to temporary variables. Those temps are then passed to both the actual call and the assumed $post call (along with the output variables after the call)

@keyboardDrummer keyboardDrummer changed the title Add contract and transparency pass Remove difference between functions and procedures in Laurel May 18, 2026
@keyboardDrummer keyboardDrummer changed the title Remove difference between functions and procedures in Laurel Merge functions and procedures in Laurel May 18, 2026
keyboardDrummer-bot and others added 2 commits May 18, 2026 12:51
…put params

Key changes:
- $post procedures are functional (isFunctional := true)
- $post takes all inputs AND all outputs as parameters
- $post does not call the original procedure — returns conjunction of
  postconditions over the combined input+output parameters
- At call sites, input arguments are assigned to temporary variables;
  those temps are passed to both the actual call and the assumed $post
  call (along with the output variables after the call)
@keyboardDrummer-bot
Copy link
Copy Markdown
Collaborator

I investigated the CI failures and pushed a fix (860a1f1) that resolves some of the test regressions. Here's a summary:

Fixed (3 tests now pass):

  • T10_ConstrainedTypes: Updated expected message from "assertion does not hold" to "precondition does not hold" (contract pass now correctly reports precondition violations).
  • T16_PropertySummary: Updated expected messages from "could not be proved" to "does not hold" (solver finds counterexamples with the new VC structure).
  • T22_MultipleReturns: Assertions now pass because I kept postconditions as free in the Opaque/Abstract body structure, so Core assumes them at call sites.
  • T19_InvokeOn: Fixed the LaurelVerifyOptions type (merge mistake — was passing Core.VerifyOptions directly).
  • DivisionByZeroCheckTest: Updated expected message for callPureDivUnsafe.

Key fix — ContractPass transformProcBody: Previously, postconditions were stripped from the body (replaced with []). This broke postcondition reasoning for calls that get lifted after the contract pass (since Core's call elimination couldn't assume them). Now postconditions are kept as free := true, meaning Core assumes them at call sites but doesn't re-check them on exit.

Remaining failures (pre-existing issues exposed by the contract/transparency pass interaction):

  1. "Core program was suppressed" bug (T2_ImpureExpressions, T8d, T14, T1_MutableFields, T2_ModifiesClauses, T5_inheritance): The lift pass's transformExpr for Assert/Assume (return expr) combined with onlyKeepSideEffectStmtsAndLast (which prepends them) results in an Unknown type reaching Core translation. This was already failing before commit 14d13ea (with a different error: "holes should have been eliminated").

  2. T19_InvokeOn — "calls to procedures are not supported in functions or contracts": After the transparency pass, functions like P, Q, R become procedures. The ensures clauses reference them in a pure context, which is disallowed. This is the fundamental issue the PR is designed to solve (as noted in the PR description).

  3. DivisionByZeroCheckTest assert z == 5: The solver can't prove pureDiv$asFunction(10, 2) == 5 despite the function having body x / y. The free postcondition z == pureDiv$asFunction(10, 2) is correctly assumed, but the solver doesn't inline the function definition. This may be related to how the SMT encoding handles define-fun for functions created by the transparency pass.

  4. T22_ArityMismatch: "input length and args length mismatch" — appears to be a separate issue in the contract pass's handling of multi-output procedures.

@keyboardDrummer-bot
Copy link
Copy Markdown
Collaborator

I pushed a fix (c6bfe97) that resolves the CI failures caused by the transparency pass incorrectly demoting functional procedures (like readField, updateField, increment) to core procedures.

Root cause: The transparency pass was only keeping contract-generated functions (with $ in name) as functions. All other functional procedures — including heap prelude functions — were treated as regular procedures, getting $asFunction copies and being demoted to isFunctional := false. This caused "calls to procedures are not supported in functions or contracts" errors when these functions were called from postconditions.

Fix: Changed the transparency pass to keep ALL isFunctional procedures as functions (not just those with $ in the name). Also fixed a bug in the contract pass where bare calls to contracted procedures in expression position would leave an assume as the block's return value instead of the output variable.

Remaining 3 test failures (T1_MutableFields, T2_ModifiesClauses, T2_ImpureExpressions) are pre-existing issues that were already failing before this commit but were masked by the readField error. They involve:

  • Unknown types from heap parametrization fresh variables reaching the translator
  • Verification regressions where modifies clause postconditions can't be proved (likely related to the $post function not being unfolded by the solver)

@keyboardDrummer-bot
Copy link
Copy Markdown
Collaborator

CI Failure Investigation

I investigated the 3 failing tests on the latest push (c6bfe97):

Failing tests

  1. T2_ImpureExpressions — "Core program was suppressed due to superfluous errors, but no diagnostics were emitted. This is a bug."
  2. T1_MutableFields — Same error
  3. T2_ModifiesClauses — Verification regression: assertions that should pass now fail with "assertion could not be proved", and assertions expected to produce "does not hold" produce "could not be proved" instead.

Root cause for T2_ImpureExpressions and T1_MutableFields

The transparency pass change in c6bfe97 moves ALL isFunctional procedures (including heap infrastructure functions like updateField, readField, increment) from coreProcedures to functions. This causes a downstream issue:

  1. After heap parameterization, field writes like c#intValue := 1 become blocks: Block [Assign [$heap, Declare $tmp] (StaticCall updateField ...), Var $tmp]
  2. These blocks are in expression position and need to be lifted to statement position
  3. The liftImperativeExpressionsInCore pass determines what to lift based on uc.coreProcedures.map (·.name.text) — since updateField is no longer in coreProcedures, the lifting behavior changes
  4. The $tmp variables end up with uniqueId=none in the translator (resolution assigns IDs but they appear to be lost somewhere in the pipeline), causing the translator to set coreProgramHasSuperfluousErrors without emitting a diagnostic

The previous filter (p.isFunctional && p.name.text.any (· == '$')) kept user-defined functions like updateField in regularProcs, which ensured they were in coreProcedures and the lifting pass worked correctly.

Root cause for T2_ModifiesClauses

Lines 45 and 107 contain assertions like assert x == d#value after calling a procedure with modifies c (not d). These should pass because the modifies clause guarantees d is unchanged. The verification regression suggests the modifies clause axioms are not being applied correctly in the current pipeline state.

Note

These 3 failures are pre-existing — they were already present at commit 860a1f1 (the commit before the last). The last commit (c6bfe97) actually reduced failures from 10 to 3. I was unable to determine a fix that doesn't reintroduce the other 7 regressions.

@keyboardDrummer keyboardDrummer force-pushed the issue-924-contract-and-proof-pass branch from c6bfe97 to 87a87cd Compare May 19, 2026 03:53
@keyboardDrummer keyboardDrummer changed the base branch from issue-21-assign-variable-type to main May 19, 2026 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants