feat: add devin provider#444
Conversation
5fb20a3 to
dc7cba9
Compare
iamtoruk
left a comment
There was a problem hiding this comment.
Thanks for this. The Devin provider itself is well-built: it follows the createXProvider / discoverSessions / createSessionParser conventions, parses defensively, opens sessions.db read-only with a static (non-interpolated) query, and is well tested. It builds clean, the full suite passes (1068 tests, 25 for Devin), the Swift menubar builds, and it merges without conflicts. So the functionality is solid. The asks below are about scope and one real cost-safety bug, so I am requesting changes and a rework into a smaller, provider-only PR.
Please remove from this PR (scope)
This PR is +1343/-6 across 22 files, and a lot of it is unrelated to adding a provider:
-
scripts/install-local-mac-menubar.sh(161 lines) + the twopackage.jsonscripts (dev:mac:menubar,local:mac:menubar). This is developer/local-build tooling, including a--system-apppath that runssudo rm -rf/sudo cp -Ragainst/Applications. It is well written, but it has nothing to do with the Devin provider and deserves its own focused review. Please split it into a separate PR (or drop it). -
tsconfig.json: remove"types": ["node"]. This is a project-wide change to ambient type resolution buried in a provider PR. We verified it is not needed: the PR builds clean with and without that line. Restricting global types to only@types/nodefor the whole tree is a forward-looking footgun (future code relying on auto-included ambient types would silently fail to typecheck). Please drop it; if a specific type collision motivated it, fix that locally and document why.
Please fix before merge (correctness / cost safety)
Validate the ACU cost before multiplying. committed_acu_cost (and the token metric fields) come from the transcript JSON and are used directly as costUSD = committedAcuCost * rate with no finiteness/sign check. Because this PR also adds devin to the set of providers whose costUSD is preserved (not recomputed in parser.ts), a non-finite, negative, or non-numeric value flows straight into aggregate spend and poisons the totals (NaN + x = NaN, or a negative spend row). The sibling mistral-vibe provider guards every untrusted number with a safeNumber() helper (typeof === 'number' && Number.isFinite && >= 0 ? v : 0); please mirror that for committed_acu_cost and the token fields, in both the hasAnyUsage gate and when building the usage object. The acuUsdRate side is already validated, so only the per-step value is exposed.
Smaller items (nice to fix)
parseNumericTimestampon a NULL / non-numericcreated_at/last_activity_atin sessions.db can throw inside the row loop, and the surrounding try/catch then drops enrichment for ALL remaining rows. Guard the parse (skip non-finite) or wrap per-row.- Dedup key
devin:<sessionId>:<step_id>: a duplicate or missingstep_idin one transcript silently drops later steps. Consider a line-index fallback whenstep_idis absent. booleanValue(v) => v === 1is only used by Devin for thehiddencolumn. Prefer inlining it in devin.ts rather than exporting a generic helper from the shared sqlite module;=== 1is also brittle ifhiddenis ever non-0/1.- Add a
CODEBURN_DEVIN_DIRenv override for the data dir, for parity with the other providers (mux, codex, etc. all support one). - Typo:
DEVIN_TRASNCRIPTS_SUBDIR->DEVIN_TRANSCRIPTS_SUBDIR. cachedFileNeedsProviderReparsereturnstrueunconditionally for Devin, which disables the cache for the provider (every run reparses every transcript). The rationale (sessions.db enrichment is not in the transcript fingerprint) is valid, but consider folding the sessions.db mtime into the fingerprint so the cache can still short-circuit when nothing changed.
In short: trim this to a provider-only change, drop the tsconfig line, add the cost-value guard, and it is in good shape.
Summary
This PR adds devin provider. Devin uses ACUs (Agent Compute Units) on costs. Since the convertion rate to USD depends on the plan we have made it configurable. Also it is mandatory to be configured in order for Devin to work as a provider. When it is not configured, we don't yield any session, hence disabling Devin from showing.
Testing
npm testpassesnpm run buildsucceedsFor new providers only:
npm run dev -- todayshows correct costs and session counts for this providernpm run dev -- models --provider <name>shows correct model names and pricing