Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,24 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-

---

## [0.143.0] - 2026-08-03 - exact campaign cost provenance

### Changed

- **Breaking:** `CampaignCellResult.costEstimated` is replaced by the required `costProvenance` union, and `ProfileSummary.totalCostUsd` becomes `number | null` with an adjacent `costProvenance` value.
- Campaign measurement digests now include the complete cost-provenance value, so digest equality is intentionally not preserved across the `0.142.x` to `0.143.0` data boundary.
- Run-record raw cost fields now describe the total as observed, estimated, or uncaptured, while `cost_known_subtotal_usd` retains any measured subtotal when the total remains unknown.
- `planCampaignRun` reports `cellsBlocked`, and each planned cell may be `blocked` when saved data cannot prove its cost or receipt identity.
- Caches created before `0.143.0` have no complete cost provenance and therefore stop a resumable campaign by default.
Inspect the plan and set `rerunInvalidCachedCells: true` for a one-time selective rerun, or set `resumable: false` for an intentional full rerun.

### Fixed

- Failed provider calls retain input, cached-input, cache-write, output, and reasoning tokens without presenting an unknown USD total as zero.
- Campaign execution inspects every saved cell and its exact billing receipts before any concurrent dispatch begins, so a late stale cell cannot waste spend in earlier cells.
- Malformed, unreadable, mismatched, or receipt-incomplete saved cells require explicit rerun approval instead of being silently reused or re-executed.
- `planCampaignRun` no longer creates the run directory while performing read-only inspection.

## [0.142.2] - 2026-08-02 - current shared type cohort

### Changed
Expand Down
2 changes: 1 addition & 1 deletion clients/python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "agent-eval-rpc"
version = "0.142.2"
version = "0.143.0"
description = "Python RPC client, official optimizer bridge, and DSPy metric adapter for @tangle-network/agent-eval."
readme = "README.md"
requires-python = ">=3.10"
Expand Down
2 changes: 1 addition & 1 deletion clients/python/src/agent_eval_rpc/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
try:
__version__ = version("agent-eval-rpc")
except PackageNotFoundError:
__version__ = "0.142.2"
__version__ = "0.143.0"

__all__ = [
"Client",
Expand Down
2 changes: 1 addition & 1 deletion clients/python/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@tangle-network/agent-eval",
"version": "0.142.2",
"version": "0.143.0",
"description": "Evaluate and improve AI agents from runs, traces, judges, and feedback. Compare candidates, cluster failures, measure lift, and gate releases.",
"homepage": "https://github.com/tangle-network/agent-eval#readme",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion src/analyst/benchmark-implementation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ export const ANALYST_BENCHMARK_DEPENDENCY_LOCK_FILES = Object.freeze([
])

export const ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256 =
'55744d067f70ffcb0223eae6121739021a9e2567208cfd14098582bb73cdbf97'
'2867725e505a600f4f1f9d934fb797f5fb94a71c906577d05009aecc254f85f3'

/** The published benchmark evidence was produced at this package version, by
* the retired one-shot direct runner, before trace analysts moved to the
Expand Down
Loading