Skip to content

Introduce recursive minimum fee calculation #1106

Merged
Jimbo4350 merged 12 commits intomasterfrom
jordan/introduce-calculate-min-fee-recursive
Mar 9, 2026
Merged

Introduce recursive minimum fee calculation #1106
Jimbo4350 merged 12 commits intomasterfrom
jordan/introduce-calculate-min-fee-recursive

Conversation

@Jimbo4350
Copy link
Contributor

Changelog

- description: |
    Introduce recursive minimum fee calculation 
  type:
  - feature     
  projects:
   - cardano-api

Context

Additional context for the PR goes here. If the PR fixes a particular issue please provide a link to the issue.

How to trust this PR

Highlight important bits of the PR that will make the review faster. If there are commands the reviewer can run to observe the new behavior, describe them.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. See Running tests for more details
  • Self-reviewed the diff

@Jimbo4350 Jimbo4350 force-pushed the jordan/introduce-calculate-min-fee-recursive branch 3 times, most recently from 7fd4c6e to 7f0d7a8 Compare February 12, 2026 15:28
@Jimbo4350 Jimbo4350 force-pushed the jordan/introduce-calculate-min-fee-recursive branch from f42c341 to 93aa7a8 Compare February 20, 2026 18:12
@Jimbo4350 Jimbo4350 marked this pull request as ready for review February 20, 2026 18:12
@Jimbo4350 Jimbo4350 force-pushed the jordan/introduce-calculate-min-fee-recursive branch 5 times, most recently from ab0d8fa to 8fabff7 Compare February 23, 2026 14:45
@carbolymer carbolymer requested a review from Copilot February 23, 2026 15:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new experimental API function to compute a minimum transaction fee via a recursive “set fee → rebalance → repeat” approach, and adds property tests intended to validate the behavior.

Changes:

  • Added calcMinFeeRecursive and RecursiveFeeCalculationError to the experimental fee API and re-exported them from Cardano.Api.Experimental.
  • Implemented recursive min-fee calculation and a helper to distribute surplus balance into outputs.
  • Added Hedgehog properties covering “well-funded succeeds”, “idempotent fixpoint”, and “insufficient funds fails”, plus updated an existing fee comparison test.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
cardano-api/src/Cardano/Api/Experimental/Tx/Internal/Fee.hs Adds the recursive fee calculation implementation and its error type.
cardano-api/src/Cardano/Api/Experimental.hs Re-exports the new recursive fee API from the experimental surface.
cardano-api/test/cardano-api-test/Test/Cardano/Api/Experimental.hs Adds new property tests for calcMinFeeRecursive and extends an existing fee test.
cardano-api/cardano-api.cabal Adds test-suite dependencies used by the new tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +688 to +705
| minFee == txBodyFee && L.isZero txBalanceCoin =
-- We have reached the minimum fee but there isn't a guarantee that
-- the inputs/outputs are balanced
return unSignTx
| minFee == txBodyFee && txBalanceCoin > 0 = do
-- We have a surplus balance so we modify the outputs to include it.
balancedOuts <- balanceTxOuts txBalanceValue unSignTx
let updatedTx = UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ Seq.fromList balancedOuts)
in return updatedTx
| txBalanceCoin < 0 = Left $ NotEnoughAda txBalanceCoin
| otherwise =
let newTx = UnsignedTx (ledgerTx & L.bodyTxL . L.feeTxBodyL .~ minFee)
in calcMinFeeRecursive newTx utxo pparams nExtraWitnesses
where
minFee = obtainCommonConstraints (useEra @era) $ L.calcMinFeeTx utxo pparams ledgerTx nExtraWitnesses
txBodyFee = ledgerTx ^. L.bodyTxL . L.feeTxBodyL
txBalanceValue = evaluateTransactionBalance pparams mempty mempty mempty utxo unSignTx
txBalanceCoin = L.coin txBalanceValue
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calcMinFeeRecursive only checks L.isZero txBalanceCoin / compares txBalanceCoin against 0, which ignores any non-ADA assets in txBalanceValue. A tx can have coin == 0 while being unbalanced in multi-assets. Consider checking L.isZero txBalanceValue and handling negative (or positive) non-ADA balance explicitly (and/or extending the error type accordingly).

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@palas palas Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calcMinFeeRecursive only checks L.isZero txBalanceCoin / compares txBalanceCoin against 0, which ignores any non-ADA assets in txBalanceValue. A tx can have coin == 0 while being unbalanced in multi-assets. Consider checking L.isZero txBalanceValue and handling negative (or positive) non-ADA balance explicitly (and/or extending the error type accordingly).

While this is true, balancing assets is much easier than balancing Ada, as long as assets cannot be used for paying fees. So we could just balance them before starting iterating (if we balance them at all). I would actually replace L.isZero with == 0

Comment on lines +693 to +696
-- We have a surplus balance so we modify the outputs to include it.
balancedOuts <- balanceTxOuts txBalanceValue unSignTx
let updatedTx = UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ Seq.fromList balancedOuts)
in return updatedTx
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the surplus case (minFee == txBodyFee && txBalanceCoin > 0), the code updates outputs and returns immediately. Updating output values can change the serialized tx size and therefore the minimum fee, potentially making the returned tx underfunded. It would be safer to rerun the fee/balance step on updatedTx (i.e. continue recursion until both fee and full balance reach a fixpoint).

Suggested change
-- We have a surplus balance so we modify the outputs to include it.
balancedOuts <- balanceTxOuts txBalanceValue unSignTx
let updatedTx = UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ Seq.fromList balancedOuts)
in return updatedTx
-- We have a surplus balance so we modify the outputs to include it,
-- then recurse to ensure the fee/balance reach a fixpoint.
balancedOuts <- balanceTxOuts txBalanceValue unSignTx
let updatedTx = UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ Seq.fromList balancedOuts)
calcMinFeeRecursive updatedTx utxo pparams nExtraWitnesses

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@palas palas Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the surplus case (minFee == txBodyFee && txBalanceCoin > 0), the code updates outputs and returns immediately. Updating output values can change the serialized tx size and therefore the minimum fee, potentially making the returned tx underfunded. It would be safer to rerun the fee/balance step on updatedTx (i.e. continue recursion until both fee and full balance reach a fixpoint).

Yes. Also, if we are not iterating both fee and balance at the same time, it would simplify the logic to separate the iteration over fee from the iteration over balance completely: here we are checking that minFee == txBodyFee and that txBalanceCoin > 0 at the same time, for example

Comment on lines +191 to +200
oldFees H.=== L.Coin 236

newFees H.=== L.Coin 236

-- Recursive fee calculation appears result in fees that are ~ 20% lower
recFee H.=== L.Coin 193

H.assert $ recFee < oldFees

H.assert $ recFee < newFees
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These assertions hard-code exact fee values (Coin 236 / Coin 193). This makes the test brittle to any future change in protocol parameters, CBOR encoding, or ledger fee calculation. Prefer asserting relationships/invariants (e.g. oldFees == newFees, and recursive fee is a fixpoint / <= non-recursive fee) rather than pinning to specific constants.

Copilot uses AI. Check for mistakes.
prop_calcMinFeeRecursive_fee_fixpoint = H.property $ do
(unsignedTx, utxo) <- H.forAll $ genFundedSimpleTx Exp.ConwayEra
case Exp.calcMinFeeRecursive unsignedTx utxo exampleProtocolParams 0 of
Left _ -> H.success
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This property currently treats a failure of calcMinFeeRecursive as success (Left _ -> H.success), which can hide real regressions. Since the generator produces a well-funded transaction, the test should fail on Left (and annotate the error) rather than passing.

Suggested change
Left _ -> H.success
Left err -> H.annotateShow err >> H.failure

Copilot uses AI. Check for mistakes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, but a comment with clarification would be nice.

minFee = obtainCommonConstraints (useEra @era) $ L.calcMinFeeTx utxo pparams ledgerTx nExtraWitnesses
txBodyFee = ledgerTx ^. L.bodyTxL . L.feeTxBodyL
txBalanceValue = evaluateTransactionBalance pparams mempty mempty mempty utxo unSignTx
txBalanceCoin = L.coin txBalanceValue
Copy link
Contributor

@carbolymer carbolymer Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about non-ada assets? Those have to be balanced as well.

Comment on lines +689 to +690
-- We have reached the minimum fee but there isn't a guarantee that
-- the inputs/outputs are balanced
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is not clear. What does it mean that there is no guarantee? Could you expand it?

Copy link
Contributor

@palas palas Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is not clear. What does it mean that there is no guarantee? Could you expand it?

Yes, do you mean other than Ada? Ada should be balanced because txBalanceCoin == 0, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the author meant non-ada assets. This should be explained.

balanceTxOuts txBalance (UnsignedTx tx) =
let outs = toList $ tx ^. L.bodyTxL . L.outputsTxBodyL
in case List.uncons outs of
Nothing -> Left NoTxOuts
Copy link
Contributor

@carbolymer carbolymer Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could be a valid case, no? I imagine user could ask to balance a minting transaction with no outputs to just get one output generated: with a change.

Copy link
Contributor Author

@Jimbo4350 Jimbo4350 Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right here. Let me push a change.

in case List.uncons outs of
Nothing -> Left NoTxOuts
Just (h, rest) ->
let updatedout = h & L.valueTxOutL %~ (<> txBalance)
Copy link
Contributor

@carbolymer carbolymer Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you simply updating the first output with the balance difference? Shouldn't you add a change output at the end instead?

Copy link
Contributor

@carbolymer carbolymer Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose that's the assumption of the function that it sends surplus change to the first output - but it's not explained anywhere. I understand that this also saves on the fees. I am not sure that everyone would want that. We should provide an option to return the change to the change address.

where
minFee = obtainCommonConstraints (useEra @era) $ L.calcMinFeeTx utxo pparams ledgerTx nExtraWitnesses
txBodyFee = ledgerTx ^. L.bodyTxL . L.feeTxBodyL
txBalanceValue = evaluateTransactionBalance pparams mempty mempty mempty utxo unSignTx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about transactions registering and deregistering stake?

-> Int
-- ^ Number of extra key hashes for native scripts
-> Either RecursiveFeeCalculationError (UnsignedTx (LedgerEra era))
calcMinFeeRecursive unSignTx@(UnsignedTx ledgerTx) utxo pparams nExtraWitnesses
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haddocks missing

Copy link
Contributor

@carbolymer carbolymer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I (w/ help of Claude) have found a counterexample for the idempotence test. See my latest commit REMOVEME: counterexample.

I think Copilot is right here: https://github.com/IntersectMBO/cardano-api/pull/1106/files#r2841432882

prop_calcMinFeeRecursive_fee_fixpoint = H.property $ do
(unsignedTx, utxo) <- H.forAll $ genFundedSimpleTx Exp.ConwayEra
case Exp.calcMinFeeRecursive unsignedTx utxo exampleProtocolParams 0 of
Left _ -> H.success
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, but a comment with clarification would be nice.

Copy link
Contributor

@palas palas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit to recursion is very nice. I have some suggestions, and I think I found a bug with help of Claude. Also I am still investigating one edge case that I think may also be an issue, but I am finding it hard to prove empirically

Comment on lines +777 to +779
| multiAssetIsNegative =
-- Case 1
Left $ NonAdaAssetsUnbalanced (getMultiAssets (useEra @era) txBalanceValue)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be iterated on? If I understand correctly it could just be checked once before iterating, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch it does not.

Comment on lines +799 to +801
multiAssetIsNegative =
obtainCommonConstraints (useEra @era) $
not (L.pointwise (>=) txBalanceValue (L.inject txBalanceCoin))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to explain why this is the same as checking that multi assets (and only multi-assets) are negative, it is not obvious at all

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be more self-documenting to compare only multi-assets to zero

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed 👍

Comment on lines +836 to +842
| lastOut ^. L.addrTxOutL == changeAddr ->
-- Update existing change output in place
let updatedOut = lastOut & L.valueTxOutL %~ (<> txBalance)
changeCoin = L.coin (updatedOut ^. L.valueTxOutL)
in if changeCoin < 0
then Left $ NotEnoughAda changeCoin
else Right $ rest Seq.:|> updatedOut
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| lastOut ^. L.addrTxOutL == changeAddr ->
-- Update existing change output in place
let updatedOut = lastOut & L.valueTxOutL %~ (<> txBalance)
changeCoin = L.coin (updatedOut ^. L.valueTxOutL)
in if changeCoin < 0
then Left $ NotEnoughAda changeCoin
else Right $ rest Seq.:|> updatedOut
| lastOut ^. L.addrTxOutL == changeAddr ->
let currentValue = lastOut ^. L.valueTxOutL
newValue = currentValue <> txBalance
changeCoin = L.coin newValue
in if changeCoin < 0
then Left $ NotEnoughAda changeCoin
else Right $ rest Seq.:|> (lastOut & L.valueTxOutL .~ newValue)

Apparently setting a negative value into the TxOut (LedgerEra era) crashes, so it needs to be checked before setting it. This was something Claude discovered when investigating a different issue.
It produces this:

│ Illegal Value in TxOut: MaryValue (Coin (-21)) (MultiAsset (fromList []))

See the also Claude generated test here: 842edd7

Check the computed value for negativity before writing it into the
TxOut. Previously the lens setter (%~) would trigger the ledger's
TxOut invariant check and throw an exception before the changeCoin < 0
guard could run. Now we compute the new value on a plain MaryValue
first, check it, and only write it with (.~) if non-negative.

Also add HasCallStack to balanceTxOuts for better error diagnostics.
@Jimbo4350 Jimbo4350 force-pushed the jordan/introduce-calculate-min-fee-recursive branch from 862bee6 to 8fe5e26 Compare March 5, 2026 18:19
Copy link
Contributor

Copilot AI commented Mar 6, 2026

@carbolymer I've opened a new pull request, #1119, to work on those changes. Once the pull request is ready, I'll request review from you.

let outs = tx ^. L.bodyTxL . L.outputsTxBodyL
in case outs of
rest Seq.:|> lastOut
| lastOut ^. L.addrTxOutL == changeAddr ->
Copy link
Contributor

@carbolymer carbolymer Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I create outputs: ["addr1", "addr2", "addr3"], and ask to send change to addr1? I think this could be improved to look up the change address here from all addresses in the outputs and modify it.

Unless this is expected to have multiple outputs pointing to the same address. If that is the case, users may be surprised that the last output gets modified with change value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're crystal-ball gazing a bit here. It depends on what we think the user wants to do. A user might create a specific output to addr1 and still want the transaction balance in a separate change output at addr1. My gut says: if a user is calling an automatic balancing function, knowing it creates a change output, and they explicitly construct an output to the same address, they likely want that output to have a specific value, with an additional change output generated separately.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're crystal-ball gazing a bit here. It depends on what we think the user wants to do.

Yes a bit. My point was to just bring this possible use case to your attention.

My gut says: if a user is calling an automatic balancing function, knowing it creates a change output, and they explicitly construct an output to the same address, they likely want that output to have a specific value, with an additional change output generated separately.

Fine by me. 👍🏻 Just wanted to be sure that this is by design.

-- transaction. These are counted as refunds on the consumed side.
-> Int
-- ^ Number of extra key hashes for native scripts
-> Either FeeCalculationError (UnsignedTx (LedgerEra era))
Copy link
Contributor

@carbolymer carbolymer Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make FeeCalculationError constructors carry HasCallStack at this point. It would make pinpointing of an error easier. For example NotEnoughAda is returned in two cases:

  1. when modifying the output
  2. when adding new output.

So it's not super obvious where did error come from, until you analyse further.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch but I'll modify the names to make it more obvious.

-> Map (Ledger.Credential Ledger.DRepRole) L.Coin
-> Int
-> Either FeeCalculationError (UnsignedTx (LedgerEra era))
go 0 _ _ _ _ _ _ _ = Left FeeCalculationDidNotConverge
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this actually an error? I think the result of the calculation can still be useful if the fee is somewhat reasonable.

@copilot figure out a definition of a "reasonable fee" which could be implemented here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is a bit hand-wavy, but not too unreasonable #1120

Copy link
Contributor

Copilot AI commented Mar 6, 2026

@carbolymer I've opened a new pull request, #1120, to work on those changes. Once the pull request is ready, I'll request review from you.

Comment on lines +803 to +804
let updatedTx = UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ balancedOuts)
go (n - 1) updatedTx utxo' pparams' poolids' stakeDelegDeposits' drepDelegDeposits' nExtraWitnesses'
Copy link
Contributor

@carbolymer carbolymer Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let updatedTx = UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ balancedOuts)
go (n - 1) updatedTx utxo' pparams' poolids' stakeDelegDeposits' drepDelegDeposits' nExtraWitnesses'
go (n - 1) $ UnsignedTx (ledgerTx & L.bodyTxL . L.outputsTxBodyL .~ balancedOuts)

Comment on lines +807 to +808
let newTx = UnsignedTx (ledgerTx & L.bodyTxL . L.feeTxBodyL .~ minFee)
in go (n - 1) newTx utxo' pparams' poolids' stakeDelegDeposits' drepDelegDeposits' nExtraWitnesses'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let newTx = UnsignedTx (ledgerTx & L.bodyTxL . L.feeTxBodyL .~ minFee)
in go (n - 1) newTx utxo' pparams' poolids' stakeDelegDeposits' drepDelegDeposits' nExtraWitnesses'
go (n - 1) $ UnsignedTx (ledgerTx & L.bodyTxL . L.feeTxBodyL .~ minFee)

(first show . Api.deserialiseFromRawBytesHex . Text.encodeUtf8)

-- ---------------------------------------------------------------------------
-- Property tests for calcMinFeeRecursive
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is too long at this point. This change should be moved to a separate module.

Copy link
Contributor Author

@Jimbo4350 Jimbo4350 Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll follow up in a separate PR: #1121

Comment on lines +828 to +830
Exp.obtainCommonConstraints era $
Exp.TxOut $
Exp.obtainCommonConstraints era $
Copy link
Contributor

@carbolymer carbolymer Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are those doubled constraints necessary? same in line 834

…constraints

- Remove getMultiAssets helper and inline the MaryValue extraction, since
  EraCommonConstraints guarantees Value ~ MaryValue for all eras including
  Dijkstra
- Remove unnecessary parameters from go's recursive loop — utxo, pparams,
  poolids, stakeDelegDeposits, drepDelegDeposits, and nExtraWitnesses are
  never modified and are already in scope from the enclosing function
- Remove redundant obtainCommonConstraints calls in test generators where
  constraints are already in scope from the outer call
@Jimbo4350 Jimbo4350 force-pushed the jordan/introduce-calculate-min-fee-recursive branch from a994f03 to 1b5f63a Compare March 9, 2026 15:27
@Jimbo4350 Jimbo4350 enabled auto-merge March 9, 2026 15:27
@Jimbo4350 Jimbo4350 added this pull request to the merge queue Mar 9, 2026
Merged via the queue into master with commit 9c85210 Mar 9, 2026
32 checks passed
@Jimbo4350 Jimbo4350 deleted the jordan/introduce-calculate-min-fee-recursive branch March 9, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants