Skip to content

cost subcommand#267

Draft
Cedric / ViaDézo1er (viadezo1er) wants to merge 16 commits into
mainfrom
cedric/cost-subcommand
Draft

cost subcommand#267
Cedric / ViaDézo1er (viadezo1er) wants to merge 16 commits into
mainfrom
cedric/cost-subcommand

Conversation

@viadezo1er

@viadezo1er Cedric / ViaDézo1er (viadezo1er) commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

STACKED BARS for bt cost --group-by user --group-by hour

say the cost is only for tokens not for storage cost

create a skill to teach the agent to use btql to see the data however they want, because we're not able to think of all the ways customers will want to use our data

TODO

repeated --group-by
too few abscissa legends in images
legends too small in images

Rebased on cedric/prune-profiles to dogfood.
#274 will be merged before this pr.

bt cost logs:

7 cost groups in BT Staging / cedric-traces
2026-07-14T01:35:49Z to 2026-07-21T01:35:49Z

Model                        Purpose    Cost       Coverage     Sources                          
claude-opus-4-8              default    $48.40     430/430      Braintrust 430                   
zai-org/GLM-5.2              default    $45.49     2267/2267    Braintrust 2267, no usage 272    
gpt-5.5                      default    $32.57     705/705      Braintrust 705, no usage 9       
gpt-5.6-sol                  default    $14.97     741/741      Braintrust 741, no usage 11      
claude-haiku-4-5-20251001    default    $0.339     50/50        Braintrust 50                    
<missing>                    default    unknown    0/4          —                                
glm-5.2                      default    unknown    0/1749       no usage 11                      

Total: ~$141.76  1753 unpriced token spans  303 without cost or usage

bt cost logs --json|jq:

{
  "project": "cedric-traces",
  "org": "BT Staging",
  "currency": "USD",
  "since": "2026-07-14T01:35:29Z",
  "until": "2026-07-21T01:35:29Z",
  "excludes_scorers": false,
  "pricing_file": null,
  "rows": [
    {
      "model": "claude-opus-4-8",
      "purpose": null,
      "candidate_spans": 430,
      "cost": 48.40370225,
      "braintrust_cost": 48.40370225,
      "file_cost": null,
      "braintrust_priced_spans": 430,
      "file_priced_spans": 0,
      "unpriced_token_spans": 0,
      "no_usage_spans": 0,
      "coverage": "full"
    },
    {
      "model": "zai-org/GLM-5.2",
      "purpose": null,
      "candidate_spans": 2539,
      "cost": 45.48671468,
      "braintrust_cost": 45.48671468,
      "file_cost": null,
      "braintrust_priced_spans": 2267,
      "file_priced_spans": 0,
      "unpriced_token_spans": 0,
      "no_usage_spans": 272,
      "coverage": "full"
    },
    {
      "model": "gpt-5.5",
      "purpose": null,
      "candidate_spans": 714,
      "cost": 32.5653,
      "braintrust_cost": 32.5653,
      "file_cost": null,
      "braintrust_priced_spans": 705,
      "file_priced_spans": 0,
      "unpriced_token_spans": 0,
      "no_usage_spans": 9,
      "coverage": "full"
    },
    {
      "model": "gpt-5.6-sol",
      "purpose": null,
      "candidate_spans": 752,
      "cost": 14.96779075,
      "braintrust_cost": 14.96779075,
      "file_cost": null,
      "braintrust_priced_spans": 741,
      "file_priced_spans": 0,
      "unpriced_token_spans": 0,
      "no_usage_spans": 11,
      "coverage": "full"
    },
    {
      "model": "claude-haiku-4-5-20251001",
      "purpose": null,
      "candidate_spans": 50,
      "cost": 0.33857415,
      "braintrust_cost": 0.33857415,
      "file_cost": null,
      "braintrust_priced_spans": 50,
      "file_priced_spans": 0,
      "unpriced_token_spans": 0,
      "no_usage_spans": 0,
      "coverage": "full"
    },
    {
      "model": null,
      "purpose": null,
      "candidate_spans": 4,
      "cost": null,
      "braintrust_cost": null,
      "file_cost": null,
      "braintrust_priced_spans": 0,
      "file_priced_spans": 0,
      "unpriced_token_spans": 4,
      "no_usage_spans": 0,
      "coverage": "unknown"
    },
    {
      "model": "glm-5.2",
      "purpose": null,
      "candidate_spans": 1760,
      "cost": null,
      "braintrust_cost": null,
      "file_cost": null,
      "braintrust_priced_spans": 0,
      "file_priced_spans": 0,
      "unpriced_token_spans": 1749,
      "no_usage_spans": 11,
      "coverage": "unknown"
    }
  ],
  "totals": {
    "candidate_spans": 6249,
    "cost": 141.76208183,
    "braintrust_cost": 141.76208183,
    "file_cost": null,
    "braintrust_priced_spans": 4193,
    "file_priced_spans": 0,
    "unpriced_token_spans": 1753,
    "no_usage_spans": 303,
    "coverage": "partial"
  }
}

@viadezo1er Cedric / ViaDézo1er (viadezo1er) changed the title cedric/cost subcommand cost subcommand Jul 10, 2026
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Latest downloadable build artifacts for this PR commit 0c407076eb84:

Available artifact names
  • artifacts-build-global
  • artifacts-build-local-x86_64-apple-darwin
  • artifacts-build-local-aarch64-pc-windows-msvc
  • artifacts-build-local-x86_64-pc-windows-msvc
  • artifacts-build-local-x86_64-unknown-linux-musl
  • artifacts-build-local-x86_64-unknown-linux-gnu
  • artifacts-build-local-aarch64-apple-darwin
  • artifacts-build-local-aarch64-unknown-linux-gnu
  • artifacts-plan-dist-manifest
  • cargo-dist-cache

first draft

BREAKING-CHANGE: --profile and --prefer-profile removed
Breaks any script using them
…n to use the new auth storage format orgid,email
The field is needed by OAuth but it's unused both bt and the backend
Therefore exposing it is useless
Now bt_cli is hardcoded as its value
@viadezo1er
Cedric / ViaDézo1er (viadezo1er) force-pushed the cedric/cost-subcommand branch 2 times, most recently from d81f21a to 0c40707 Compare July 21, 2026 01:33
 - Preserves separate OAuth and API-key login identities.
 - Keeps multiple API keys for the same org distinct using key hints.
 - Implements the intended precedence:
     1. Explicit --api-key
     2. --prefer-api-key
     3. OAuth
     4. BRAINTRUST_API_KEY
     5. Stored API key
 - Validates explicit/environment API keys against the requested org.
 - Does not fall back to OAuth when a selected key is invalid or belongs to another org.
 - Rejects API-key authentication in cross-org mode.
 - Pins the exact API-key login selected by bt switch or bt init.
 - Adds bt auth login --global/--local.
 - Adds filtering to bt auth logins --org/--prefer-api-key.
 - Changes bare bt auth logout to select from all saved logins instead of deleting the currently active one.
 - Adds bt auth logout --oauth and retains --api-key-hint.

 Config handling — src/config/mod.rs

 - Makes global/local merging org-safe:
     - A project is never inherited from another org.
     - project and project_id remain coupled.
     - Local project-without-org does not inherit the global org.

 - Stores cross-org as:

   ```json
     { "org": "" }
   ```

 - Ignores the legacy profile field.

 - Preserves unknown config keys during updates.

 - Only treats .bt/config.json as a local config; a bare .bt directory is not one.

 - Adds separate filesystem discovery rules for bt init.

 bt switch — src/switch.rs

 - Starts with a saved-login picker:
     - OAuth entries collapse into org choices.
     - API keys remain separate and display their hints.
     - Cross-org OAuth is selectable.
 - Automatically selects a sole login/project.
 - Prompts for project when multiple projects exist.
 - Pins a selected API-key slot exactly.
 - Implements global/local scope rules:
     - --global does not search for local config.
     - --local requires an existing .bt/config.json.
     - Interactive ambiguity opens a scope picker.
     - Non-interactive ambiguity errors before network work.
 - Preserves unknown config fields.
 - Fixes the scope-picker ANSI corruption you reported by passing plain labels to Dialoguer.

 bt init — src/init.rs

 - Adds:
     - --here
     - -f/--force

 - Searches upward for .bt, .git, home, or filesystem root using the specified boundary rules.

 - Resolves destination errors before authentication.

 - Uses the same saved-login/project selection behavior as switch.

 - Excludes cross-org from the picker because init requires a project.

 - Writes org, project, and project_id.

 - Reports permission/write failures with the affected path.

 - Now exposes the real destination error instead of only saying:

   ```text
     could not resolve `bt init` destination
   ```

 Cross-org CLI/status — src/args.rs, src/main.rs, src/status.rs

 - Normalizes these to canonical cross-org:

   ```sh
     --org cross-org
     --org ""
   ```

 - Displays cross-org as cross-org in human and JSON status output.

 - Prevents stale projects from being combined with cross-org context.

 - Makes --prefer-api-key fail actionably from cross-org context.
multiple group-by work
sort by time when grouped by day/hour
sort by the dimension (eg user) in the output data when grouped by user
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant