Skip to content
This repository was archived by the owner on Mar 2, 2026. It is now read-only.

CLAUDE.md

Latest commit

 

History

History
174 lines (130 loc) · 7.19 KB

CLAUDE.md

File metadata and controls

174 lines (130 loc) · 7.19 KB

CLAUDE.md

This file provides guidance for AI assistants working with the OpenProxy codebase.

Project Overview

OpenProxy is a lightweight, production-ready LLM proxy server that forwards API requests to OpenAI and Anthropic-compatible endpoints. It provides comprehensive logging, automatic cost tracking, and PostgreSQL integration. It includes a Next.js metrics dashboard for real-time analytics.

Repository Structure

openproxy/
├── proxy.ts                 # Main proxy server (Node.js http module, ~500 lines)
├── cost.ts                  # Cost calculation engine (Helicone API + custom pricing)
├── dashboard/               # Next.js metrics dashboard (separate workspace package)
│   ├── app/
│   │   ├── api/metrics/
│   │   │   └── route.ts     # Metrics API endpoint (GET /api/metrics)
│   │   ├── layout.tsx       # Root layout
│   │   └── page.tsx         # Main dashboard page (client component)
│   └── components/
│       ├── MetricsOverview.tsx
│       ├── ModelBreakdown.tsx
│       ├── RecentRequests.tsx
│       └── TrendsChart.tsx
├── package.json             # Root workspace package
├── pnpm-workspace.yaml      # Monorepo: root "." + "dashboard"
├── tsconfig.json            # ES2022, CommonJS, strict mode
├── Dockerfile               # node:22-slim, builds TS, runs dist/proxy.js
├── .env.example             # Environment variable template
└── .github/
    └── dependabot.yml       # Weekly dependency updates

Tech Stack

  • Runtime: Node.js 22
  • Language: TypeScript (strict mode, ES2022 target, CommonJS module)
  • Package Manager: pnpm (v10.26.1, monorepo workspace)
  • Proxy Server: Node.js native http module (no Express/Fastify)
  • Dashboard: Next.js 16, React 18, Recharts 3
  • Database: PostgreSQL via pg library (connection pooling)
  • Key Dependencies: dotenv, pg, uuid

Development Commands

# Install dependencies
pnpm install

# Run proxy server in development (auto-reload via ts-node-dev)
pnpm dev

# Build TypeScript to dist/
pnpm build

# Run compiled proxy server
pnpm start

# Run dashboard in development (port 3008)
cd dashboard && pnpm dev

# Build dashboard
cd dashboard && pnpm build

Environment Variables

Required in .env (see .env.example):

Variable Default Description
PORT 3007 Proxy server port
OPENAI_UPSTREAM_URL - Upstream OpenAI-compatible endpoint
ANTHROPIC_UPSTREAM_URL - Upstream Anthropic-compatible endpoint
DATABASE_URL - PostgreSQL connection string

The dashboard reads DATABASE_URL directly and runs on port 3008.

Architecture

Proxy Server (proxy.ts)

The proxy is a single-file HTTP server with this request flow:

  1. Route parsing - Extract provider from URL prefix (/openai/* or /anthropic/*)
  2. Auth extraction - Provider-specific: x-api-key for Anthropic, Authorization: Bearer for OpenAI
  3. Upstream forwarding - Uses fetch() to proxy the request to the configured upstream URL
  4. Response handling - Supports both streaming (SSE text/event-stream) and non-streaming responses
  5. Async logging - Persists request/response data to PostgreSQL without blocking the response

Key functions:

  • parseRoute() - Maps request paths to providers
  • getProviderConfig() - Retrieves upstream URL configuration
  • getAuthToken() - Extracts auth tokens with provider-specific logic
  • buildUpstreamHeaders() - Constructs provider-appropriate headers
  • normalizeUsage() - Normalizes token usage across OpenAI/Anthropic formats
  • persistDatabaseRecord() - Logs request data to the llm_proxy table

Cost Engine (cost.ts)

Calculates per-request costs in USD using token usage data:

  1. Custom costs (CUSTOM_MODEL_COSTS) - Hardcoded overrides, checked first
  2. Helicone costs - Fetched from Helicone API at startup, matched by operator priority: equals > startsWith > includes
  3. Fallback - Returns zero cost if no match found

Costs are expressed per 1M tokens. The calculateCost() function accounts for cached tokens at reduced rates.

Dashboard (dashboard/)

A standalone Next.js app that queries PostgreSQL directly for metrics:

  • Summary statistics, hourly trends, model breakdown, recent requests
  • Auto-refresh (30s intervals), time range selection (1h to 7d)
  • Server-side API route at /api/metrics with parameter validation

Key Types

type Provider = "openai" | "anthropic"

interface NormalizedUsage {
  prompt_tokens: number | null
  completion_tokens: number | null
  total_tokens: number | null
  cached_tokens: number | null
}

type CostConfig = {
  input: number    // USD per 1M prompt tokens
  cached: number   // USD per 1M cached tokens
  output: number   // USD per 1M completion tokens
}

Database Schema

The proxy logs to a llm_proxy table with columns including: timestamp, request_method, request_path, provider, model, token counts (prompt_tokens, completion_tokens, total_tokens, cached_tokens), total_cost, response_time, full request_body/response_body JSON, response_status, client_ip, user_agent, request_size, response_size, stream, temperature, max_tokens, and request_id (UUID).

Error Handling Conventions

  • 404 - Invalid provider prefix (INVALID_PROVIDER_PREFIX)
  • 401 - Missing auth token (MISSING_OR_INVALID_AUTHORIZATION_HEADER)
  • 503 - Upstream URL not configured (UPSTREAM_URL_NOT_CONFIGURED)
  • 502 - Upstream connection failure or internal error (UPSTREAM_CONNECTION_FAILED, INTERNAL_SERVER_ERROR)
  • Database errors are logged to console but never block the response to the client

Code Conventions

  • No test framework - No tests exist yet; no Jest/Vitest configured
  • No linter/formatter - No ESLint or Prettier configured at the root level
  • Strict TypeScript - strict: true in tsconfig
  • Async/await throughout; database logging uses fire-and-forget with .catch()
  • Console logging uses ANSI color codes for readability (green for success, yellow for warnings, etc.)
  • Parameterized SQL queries - All database queries use $1, $2, ... placeholders to prevent injection
  • Numeric parsing uses Number.parseInt() / Number.parseFloat() / Number.isNaN() (not global equivalents)
  • Commit messages follow conventional commits loosely (feat:, fix:, chore:, refactor:)

Docker

docker build -t openproxy .
docker run -p 8080:8080 --env-file .env openproxy

The Dockerfile builds TypeScript inside the container and runs the compiled dist/proxy.js. Default port inside the container is 8080.

Important Notes

  • The proxy passes auth tokens through without validation - it trusts whatever the client provides
  • Streaming responses (SSE) are piped chunk-by-chunk to maintain real-time delivery
  • The dashboard is a separate workspace package with its own package.json and tsconfig.json
  • No external HTTP framework is used; the proxy is built entirely on node:http and fetch()
  • The @types/uuid package is incorrectly listed under dependencies instead of devDependencies in the root package.json