Skip to content

feat: add devin provider#444

Open
tvcsantos wants to merge 3 commits into
getagentseal:mainfrom
tvcsantos:feature/add-devin-provider
Open

feat: add devin provider#444
tvcsantos wants to merge 3 commits into
getagentseal:mainfrom
tvcsantos:feature/add-devin-provider

Conversation

@tvcsantos
Copy link
Copy Markdown

@tvcsantos tvcsantos commented Jun 5, 2026

Summary

This PR adds devin provider. Devin uses ACUs (Agent Compute Units) on costs. Since the convertion rate to USD depends on the plan we have made it configurable. Also it is mandatory to be configured in order for Devin to work as a provider. When it is not configured, we don't yield any session, hence disabling Devin from showing.

Testing

  • I have tested this locally against real data (not just unit tests)
  • npm test passes
  • npm run build succeeds

For new providers only:

  • I installed the tool and generated real sessions by using it
  • npm run dev -- today shows correct costs and session counts for this provider
  • npm run dev -- models --provider <name> shows correct model names and pricing
  • Screenshot or terminal output attached below proving it works with real data

@tvcsantos tvcsantos force-pushed the feature/add-devin-provider branch from 5fb20a3 to dc7cba9 Compare June 6, 2026 13:22
Copy link
Copy Markdown
Member

@iamtoruk iamtoruk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The Devin provider itself is well-built: it follows the createXProvider / discoverSessions / createSessionParser conventions, parses defensively, opens sessions.db read-only with a static (non-interpolated) query, and is well tested. It builds clean, the full suite passes (1068 tests, 25 for Devin), the Swift menubar builds, and it merges without conflicts. So the functionality is solid. The asks below are about scope and one real cost-safety bug, so I am requesting changes and a rework into a smaller, provider-only PR.

Please remove from this PR (scope)

This PR is +1343/-6 across 22 files, and a lot of it is unrelated to adding a provider:

  1. scripts/install-local-mac-menubar.sh (161 lines) + the two package.json scripts (dev:mac:menubar, local:mac:menubar). This is developer/local-build tooling, including a --system-app path that runs sudo rm -rf / sudo cp -R against /Applications. It is well written, but it has nothing to do with the Devin provider and deserves its own focused review. Please split it into a separate PR (or drop it).

  2. tsconfig.json: remove "types": ["node"]. This is a project-wide change to ambient type resolution buried in a provider PR. We verified it is not needed: the PR builds clean with and without that line. Restricting global types to only @types/node for the whole tree is a forward-looking footgun (future code relying on auto-included ambient types would silently fail to typecheck). Please drop it; if a specific type collision motivated it, fix that locally and document why.

Please fix before merge (correctness / cost safety)

Validate the ACU cost before multiplying. committed_acu_cost (and the token metric fields) come from the transcript JSON and are used directly as costUSD = committedAcuCost * rate with no finiteness/sign check. Because this PR also adds devin to the set of providers whose costUSD is preserved (not recomputed in parser.ts), a non-finite, negative, or non-numeric value flows straight into aggregate spend and poisons the totals (NaN + x = NaN, or a negative spend row). The sibling mistral-vibe provider guards every untrusted number with a safeNumber() helper (typeof === 'number' && Number.isFinite && >= 0 ? v : 0); please mirror that for committed_acu_cost and the token fields, in both the hasAnyUsage gate and when building the usage object. The acuUsdRate side is already validated, so only the per-step value is exposed.

Smaller items (nice to fix)

  • parseNumericTimestamp on a NULL / non-numeric created_at / last_activity_at in sessions.db can throw inside the row loop, and the surrounding try/catch then drops enrichment for ALL remaining rows. Guard the parse (skip non-finite) or wrap per-row.
  • Dedup key devin:<sessionId>:<step_id>: a duplicate or missing step_id in one transcript silently drops later steps. Consider a line-index fallback when step_id is absent.
  • booleanValue(v) => v === 1 is only used by Devin for the hidden column. Prefer inlining it in devin.ts rather than exporting a generic helper from the shared sqlite module; === 1 is also brittle if hidden is ever non-0/1.
  • Add a CODEBURN_DEVIN_DIR env override for the data dir, for parity with the other providers (mux, codex, etc. all support one).
  • Typo: DEVIN_TRASNCRIPTS_SUBDIR -> DEVIN_TRANSCRIPTS_SUBDIR.
  • cachedFileNeedsProviderReparse returns true unconditionally for Devin, which disables the cache for the provider (every run reparses every transcript). The rationale (sessions.db enrichment is not in the transcript fingerprint) is valid, but consider folding the sessions.db mtime into the fingerprint so the cache can still short-circuit when nothing changed.

In short: trim this to a provider-only change, drop the tsconfig line, add the cost-value guard, and it is in good shape.

@iamtoruk iamtoruk mentioned this pull request Jun 6, 2026
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants