Feat: Custom API Integrations

[sgogriff]

Summary

Adds API Integrations as a new telemetry subsystem for tracking token and cost usage from custom API-driven scripts via local JSONL ingestion.

What Changed

• added backend JSONL ingestion and SQLite storage for API Integrations events
• added read-only API endpoints for current usage, history, and ingest health
• added API Integrations dashboard UI and settings visibility control
• added Python wrapper examples for Anthropic, OpenAI, Mistral, OpenRouter, and Gemini
• added setup and README documentation

Notes

• API Integrations is separate from subscription/quota tracking, codebase reflects this
• it tracks cumulative usage telemetry, not remaining plan percentage
• ingestion is controlled by ONWATCH_API_INTEGRATIONS_ENABLED
• source directory is configurable via ONWATCH_API_INTEGRATIONS_DIR
• Docs currently only have a .py wrapper as an example, but more could easily be added. The same goes for adapting to a wider range of providers.

Testing

• go test -race ./...
• go vet ./...
• manually tested JSONL ingestion and dashboard rendering with seeded data. Seeded data generator script available in examples/api_integrations/python along with .py examples and JSONL wrapper.

Screenshots!

[codecov]

Codecov Report

❌ Patch coverage is 77.52404% with 187 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/store/api_integrations_store.go	80.88%	28 Missing and 24 partials ⚠️
internal/agent/api_integrations_ingest_agent.go	61.86%	26 Missing and 19 partials ⚠️
internal/web/api_integrations_handlers.go	85.71%	26 Missing and 8 partials ⚠️
internal/web/handlers.go	58.33%	18 Missing and 7 partials ⚠️
internal/api_integrations/types.go	68.83%	13 Missing and 11 partials ⚠️
internal/config/config.go	70.83%	3 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

[prakersh]

Thanks a lot for this contribution! This is a substantial PR so we'll need some time to review it properly. Will follow up with detailed feedback once we've gone through everything.

[sgogriff]

No problem! Let me know if there is anything that needs re-thinking. Happy to help :)

[prakersh]

Thanks for the detailed PR! I've done an initial review and the overall structure looks solid - clean separation, good test coverage, and consistent use of parameterized SQL.

A few things I'd like to address before merging:

Query bounds:

• QueryAPIIntegrationUsageSummary() has no LIMIT - our project guardrails require bounded queries. Would need a cap here.
• QueryAPIIntegrationUsageBuckets() loads all events into memory before grouping. For large time ranges this could be expensive - ideally the bucketing should happen in SQL.

Minor code issues:

• Duplicate detection in InsertAPIIntegrationUsageEvent relies on strings.Contains(err.Error(), "unique") which is fragile. Prefer checking SQLite error codes.
• GetActiveSystemAlertsByProvider double-parses createdAt (RFC3339 then RFC3339Nano) - second parse always overwrites the first. Just RFC3339Nano would suffice.

Typo:

• README line mentions "accross seprarte" - should be "across separate"

Worth considering (non-blocking):

• No data retention/pruning for api_integration_usage_events - will grow indefinitely
• raw_line column stores full JSON alongside parsed columns, doubling storage

Happy to discuss any of these. Nice work on the docs and Python examples.

[sgogriff]

Glad that the PR looks good - I've gone through your comments and made these changes: -

• Bounded the current usage summary query, limits to 500.
• Moved API Integrations history bucketing into SQLite so large time ranges no longer require loading and grouping all raw events in memory.
• Hardened duplicate detection for ingested events; switching from error-string matching to SQLite unique-constraint codes.
• API Integrations alert timestamp parsing by using a single RFC3339Nano parse path.
• Fixed the README typo in the API Integrations description.
• Added automatic database retention/pruning for api_integration_usage_events via ONWATCH_API_INTEGRATIONS_RETENTION.
• Default retention is now 60 days (1440h). And setting the value to 0 should disable DB pruning.
• Source API Integrations JSONL files are not pruned by onWatch (future change? would have to think about how this interacts with offset_bytes in JSONL tailing). People with lots of integrations still need to rotate file manually.
• Updated the API Integrations setup docs and README env var reference to document the new retention behaviour.

I didn't change raw_line column storing full JSON doubling storage. Probably needs addressing.

Also had a couple thoughts with where to take this next. Won't be doing it anytime soon, but any thoughts?

• Add settings tab for the API integrations, similar to the others. Make it easier for end user!
• Add some kind of threshold alerts (i.e. integration exceeds X tokens or $ cost in Y window). This one would be particularly useful.

Let me know if somethings not right, or if we should make more changes before merging :)