improvement(types): enforce patterns outside just hooks directory and fix CI check + fix tracing billing issue (#4367)

* improvement(types): enforce outside just hooks dir and update CI checks

* fix billing account details for kb embeddings

* more fixes

* fix byok issue

* address comments

* fix

* more comments

* address bugbot

Author: icecrasher321

apps/sim/AGENTS.md

@@ -5,6 +5,7 @@ These rules apply to files under `apps/sim/` in addition to the repository root
 ## Architecture
 
 ### Core Principles
+
 1. **Single Responsibility**: Each component, hook, store has one clear purpose
 2. **Composition Over Complexity**: Break down complex logic into smaller pieces
 3. **Type Safety First**: TypeScript interfaces for all props, state, return types
@@ -47,6 +48,7 @@ feature/
 ```
 
 ### Naming Conventions
+
 - **Components**: PascalCase (`WorkflowList`)
 - **Hooks**: `use` prefix (`useWorkflowOperations`)
 - **Files**: kebab-case (`workflow-list.tsx`)
@@ -71,7 +73,7 @@ feature/
 
 ## API Contracts
 
-Boundary HTTP request and response shapes for all routes under `apps/sim/app/api/**` live in `apps/sim/lib/api/contracts/**` (one file per resource family). Routes never define route-local boundary Zod schemas, and clients never define ad-hoc wire types — both sides consume the same contract.
+Boundary HTTP request and response shapes for all routes under `apps/sim/app/api/`** live in `apps/sim/lib/api/contracts/**` (one file per resource family). Routes never define route-local boundary Zod schemas, and clients never define ad-hoc wire types — both sides consume the same contract.
 
 - Each contract is built with `defineRouteContract({ method, path, params?, query?, body?, headers?, response: { mode: 'json', schema } })` from `@/lib/api/contracts`.
 - Contracts export named schemas AND named TypeScript type aliases (e.g., `export type CreateFolderBody = z.input<typeof createFolderBodySchema>`). Clients import the named aliases — never `z.input<...>` / `z.output<...>` in hooks.
@@ -83,10 +85,10 @@ Boundary HTTP request and response shapes for all routes under `apps/sim/app/api
 
 A small number of legitimate exceptions to the boundary rules are tolerated when annotated. The audit script recognizes four annotation forms:
 
-- `// boundary-raw-fetch: <reason>` — placed on the line directly above a raw `fetch(` call inside `apps/sim/hooks/queries/**` or `apps/sim/hooks/selectors/**`. Use only for documented exceptions: streaming responses, binary downloads, multipart uploads, signed-URL flows, OAuth redirects, and external-origin requests.
+- `// boundary-raw-fetch: <reason>` — placed on the line directly above a raw `fetch(` call inside `apps/sim/hooks/queries/**`, `apps/sim/hooks/selectors/**`, or any other client/UI source under `apps/sim/**` that targets a same-origin `/api/...` URL. Use only for documented exceptions: streaming responses, binary downloads, multipart uploads, signed-URL flows, OAuth redirects, and external-origin requests.
 - `// double-cast-allowed: <reason>` — placed on the line directly above an `as unknown as X` cast outside test files.
-- `// boundary-raw-json: <reason>` — placed on the line directly above a raw `await request.json()` / `await req.json()` read in a route handler. Use only when the body is a JSON-RPC envelope, a tolerant `.catch(() => ({}))` parse, or otherwise cannot go through `parseRequest`.
-- `// untyped-response: <reason>` — placed on the line directly above a `schema: z.unknown()` response declaration in a contract file. Use only when the response body is genuinely opaque (user-supplied data, third-party passthrough).
+- `// boundary-raw-json: <reason>` — placed on the line directly above a raw `await request.json()` / `await req.json()` read (or the multi-line `await request.clone().json()` shim variant) in a route handler. Use only when the body is a JSON-RPC envelope, a tolerant `.catch(() => ({}))` parse, or otherwise cannot go through `parseRequest`.
+- `// untyped-response: <reason>` — placed on the line directly above a `schema: z.unknown()` / `schema: z.object({}).passthrough()` / `schema: z.record(z.string(), z.unknown())` response declaration (or a simple alias to one of those) in a contract file. Use only when the response body is genuinely opaque (user-supplied data, third-party passthrough).
 
 Placement rule: the annotation must immediately precede the call or cast. Up to three non-empty preceding comment lines are tolerated, so additional context comments above the annotation are fine. The reason must be non-empty after trimming — annotations with empty reasons fail strict mode (`annotationsMissingReason`).
 
@@ -104,6 +106,19 @@ const response = await fetch(`/api/copilot/chat/stream?chatId=${chatId}`, { sign
 const provider = config as unknown as LegacyProvider
 ```
 
+```ts
+// boundary-raw-json: shim pre-validates the mothership envelope before delegating to the copilot handler that consumes the body
+const body = await request
+  .clone()
+  .json()
+  .catch(() => undefined)
+```
+
+```ts
+// untyped-response: forwards firecrawl /v2/parse response unchanged for downstream tool consumers
+output: z.unknown(),
+```
+
 ## API Route Pattern
 
 Routes never `import { z } from 'zod'` and never define route-local boundary schemas. They consume the contract from `@/lib/api/contracts/**` and validate with canonical helpers from `@/lib/api/server`:
@@ -149,18 +164,18 @@ When adding a new route + client surface, follow this order. Each step has one p
 
 LLMs will write contracts that compile but are sloppy. The human reviewer should optimize attention on:
 
-- **`required` vs `optional` vs `nullable` is correct**. `optional()` allows omission; `nullable()` allows `null`; chaining both creates a tri-state that's almost never what you want.
+- `**required` vs `optional` vs `nullable` is correct**. `optional()` allows omission; `nullable()` allows `null`; chaining both creates a tri-state that's almost never what you want.
 - **Response schema matches the route's actual JSON output**. The most common drift bug — route emits a field the schema doesn't declare, or omits a required field. Walk every `NextResponse.json(...)` callsite against the schema.
 - **Error messages are descriptive**. `'fileName cannot be empty'` beats `'Required'`. Use the second arg of `min(1, '...')`, `nonempty('...')`, etc. For cross-field refines, use `superRefine` with a `path` and a message that names the failing field.
 - **Bounds are set** on arrays (`.min(1)`, `.max(N)`), strings (`.min(1).max(N)` for IDs/names), and numbers (`.min().max()` for limits/sizes).
-- **`z.unknown()` is a smell** unless the data is genuinely arbitrary (provider passthrough, user-defined tool result, JSON-RPC envelope). When kept, must be annotated `// untyped-response: <specific reason>` in a `schema:` slot.
+- `**z.unknown()` is a smell** unless the data is genuinely arbitrary (provider passthrough, user-defined tool result, JSON-RPC envelope). When kept, must be annotated `// untyped-response: <specific reason>` in a `schema:` slot.
 - **Discriminated unions over plain unions** when the wire has a discriminant field — gives clients exhaustive narrowing.
 
 CI (`bun run check:api-validation:strict`) catches structural violations (Zod imports in routes, raw `request.json()`, double casts, missing annotations). It does **not** catch these schema-quality judgments — that's the human's job in PR review.
 
 ## React Query Client Boundary
 
-Hooks in `apps/sim/hooks/queries/**` consume contracts the same way routes do. Every same-origin JSON call must go through `requestJson(contract, ...)` from `@/lib/api/client/request` instead of raw `fetch`:
+Hooks in `apps/sim/hooks/queries/`** consume contracts the same way routes do. Every same-origin JSON call must go through `requestJson(contract, ...)` from `@/lib/api/client/request` instead of raw `fetch`:
 
 - Hooks import named type aliases from `@/lib/api/contracts/**`. Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code.
 - `requestJson` parses params, query, body, and headers against the contract on the way out and validates the JSON response on the way back. Hooks always forward `signal` for cancellation.
@@ -204,3 +219,4 @@ export function useEntityList(workspaceId?: string) {
 - **Create `utils.ts` when** 2+ files need the same helper
 - **Check existing sources** before duplicating (`lib/` has many utilities)
 - **Location**: `lib/` (app-wide) → `feature/utils/` (feature-scoped) → inline (single-use)
+

apps/sim/app/(auth)/login/login-form.tsx

@@ -15,6 +15,8 @@ import {
   ModalDescription,
   ModalHeader,
 } from '@/components/emcn'
+import { requestJson } from '@/lib/api/client/request'
+import { forgetPasswordContract } from '@/lib/api/contracts'
 import { client } from '@/lib/auth/auth-client'
 import { getEnv, isFalsy, isTruthy } from '@/lib/core/config/env'
 import { validateCallbackUrl } from '@/lib/core/security/input-validation'
@@ -282,20 +284,16 @@ export default function LoginPage({
       setIsSubmittingReset(true)
       setResetStatus({ type: null, message: '' })
 
-      const response = await fetch('/api/auth/forget-password', {
-        method: 'POST',
-        headers: {
-          'Content-Type': 'application/json',
-        },
-        body: JSON.stringify({
-          email: forgotPasswordEmail,
-          redirectTo: `${getBaseUrl()}/reset-password`,
-        }),
-      })
-
-      if (!response.ok) {
-        const errorData = await response.json()
-        let errorMessage = errorData.message || 'Failed to request password reset'
+      try {
+        await requestJson(forgetPasswordContract, {
+          body: {
+            email: forgotPasswordEmail,
+            redirectTo: `${getBaseUrl()}/reset-password`,
+          },
+        })
+      } catch (requestError) {
+        let errorMessage =
+          requestError instanceof Error ? requestError.message : 'Failed to request password reset'
 
         if (
           errorMessage.includes('Invalid body parameters') ||

apps/sim/app/(auth)/oauth/consent/page.tsx

@@ -5,6 +5,8 @@ import { ArrowLeftRight } from 'lucide-react'
 import Image from 'next/image'
 import { useRouter, useSearchParams } from 'next/navigation'
 import { Button, Loader } from '@/components/emcn'
+import { requestJson } from '@/lib/api/client/request'
+import { oauthAuthorizeParamsContract } from '@/lib/api/contracts/oauth-connections'
 import { signOut, useSession } from '@/lib/auth/auth-client'
 import { AUTH_SUBMIT_BTN } from '@/app/(auth)/components/auth-button-classes'
 
@@ -44,6 +46,7 @@ export default function OAuthConsentPage() {
       return
     }
 
+    // boundary-raw-fetch: better-auth catch-all OAuth client lookup, no app-level contract
     fetch(`/api/auth/oauth2/client/${encodeURIComponent(clientId)}`, { credentials: 'include' })
       .then(async (res) => {
         if (!res.ok) return
@@ -65,6 +68,7 @@ export default function OAuthConsentPage() {
 
       setSubmitting(true)
       try {
+        // boundary-raw-fetch: better-auth catch-all OAuth consent submission, no app-level contract
         const res = await fetch('/api/auth/oauth2/consent', {
           method: 'POST',
           headers: { 'Content-Type': 'application/json' },
@@ -100,15 +104,15 @@ export default function OAuthConsentPage() {
   const handleSwitchAccount = useCallback(async () => {
     if (!consentCode) return
 
-    const res = await fetch(`/api/auth/oauth2/authorize-params?consent_code=${consentCode}`, {
-      credentials: 'include',
-    })
-    if (!res.ok) {
+    const params = await requestJson(oauthAuthorizeParamsContract, {
+      query: { consent_code: consentCode },
+    }).catch(() => null)
+
+    if (!params) {
       setError('Unable to switch accounts. Please re-initiate the connection.')
       return
     }
 
-    const params = (await res.json()) as Record<string, string | null>
     const authorizeUrl = new URL('/api/auth/oauth2/authorize', window.location.origin)
     for (const [key, value] of Object.entries(params)) {
       if (value) authorizeUrl.searchParams.set(key, value)

apps/sim/app/(auth)/reset-password/reset-password-content.tsx

@@ -4,6 +4,8 @@ import { Suspense, useState } from 'react'
 import { createLogger } from '@sim/logger'
 import Link from 'next/link'
 import { useRouter, useSearchParams } from 'next/navigation'
+import { requestJson } from '@/lib/api/client/request'
+import { resetPasswordContract } from '@/lib/api/contracts'
 import { SetNewPasswordForm } from '@/app/(auth)/reset-password/reset-password-form'
 
 const logger = createLogger('ResetPasswordPage')
@@ -27,26 +29,18 @@ function ResetPasswordContent() {
     : null
 
   const handleResetPassword = async (password: string) => {
+    if (!token) return
     try {
       setIsSubmitting(true)
       setStatusMessage({ type: null, text: '' })
 
-      const response = await fetch('/api/auth/reset-password', {
-        method: 'POST',
-        headers: {
-          'Content-Type': 'application/json',
-        },
-        body: JSON.stringify({
+      await requestJson(resetPasswordContract, {
+        body: {
           token,
           newPassword: password,
-        }),
+        },
       })
 
-      if (!response.ok) {
-        const errorData = await response.json()
-        throw new Error(errorData.message || 'Failed to reset password')
-      }
-
       setStatusMessage({
         type: 'success',
         text: 'Password reset successful! Redirecting to login...',

apps/sim/app/(landing)/components/auth-modal/auth-modal.tsx

@@ -14,6 +14,8 @@ import {
   ModalTrigger,
 } from '@/components/emcn'
 import { GithubIcon, GoogleIcon } from '@/components/icons'
+import { requestJson } from '@/lib/api/client/request'
+import { type AuthProviderStatusResponse, getAuthProvidersContract } from '@/lib/api/contracts/auth'
 import { client } from '@/lib/auth/auth-client'
 import { getEnv, isFalsy, isTruthy } from '@/lib/core/config/env'
 import { captureClientEvent } from '@/lib/posthog/client'
@@ -30,13 +32,9 @@ interface AuthModalProps {
   source: PostHogEventMap['auth_modal_opened']['source']
 }
 
-interface ProviderStatus {
-  githubAvailable: boolean
-  googleAvailable: boolean
-  registrationDisabled: boolean
-}
+type ProviderStatus = AuthProviderStatusResponse
 
-let fetchPromise: Promise<ProviderStatus> | null = null
+let fetchPromise: Promise<AuthProviderStatusResponse> | null = null
 
 const FALLBACK_STATUS: ProviderStatus = {
   githubAvailable: false,
@@ -49,12 +47,8 @@ const SOCIAL_BTN =
 
 function fetchProviderStatus(): Promise<ProviderStatus> {
   if (fetchPromise) return fetchPromise
-  fetchPromise = fetch('/api/auth/providers')
-    .then((r) => {
-      if (!r.ok) throw new Error(`HTTP ${r.status}`)
-      return r.json()
-    })
-    .then(({ githubAvailable, googleAvailable, registrationDisabled }: ProviderStatus) => ({
+  fetchPromise = requestJson(getAuthProvidersContract, {})
+    .then(({ githubAvailable, googleAvailable, registrationDisabled }) => ({
       githubAvailable,
       googleAvailable,
       registrationDisabled,

apps/sim/app/(landing)/components/contact/contact-form.tsx

@@ -7,10 +7,13 @@ import { useMutation } from '@tanstack/react-query'
 import Link from 'next/link'
 import { Combobox, Input, Textarea } from '@/components/emcn'
 import { Check } from '@/components/emcn/icons'
+import { requestJson } from '@/lib/api/client/request'
 import {
   CONTACT_TOPIC_OPTIONS,
   type ContactRequestPayload,
   contactRequestSchema,
+  type SubmitContactBody,
+  submitContactContract,
 } from '@/lib/api/contracts/contact'
 import { flattenFieldErrors } from '@/lib/api/contracts/primitives'
 import { getEnv } from '@/lib/core/config/env'
@@ -53,29 +56,8 @@ const LANDING_SUBMIT =
 const LANDING_LABEL =
   'font-[500] font-season text-[13px] text-[var(--landing-text)] tracking-[0.02em]'
 
-interface SubmitContactRequestInput extends ContactRequestPayload {
-  website: string
-  captchaToken?: string
-  captchaUnavailable?: boolean
-}
-
-async function submitContactRequest(payload: SubmitContactRequestInput) {
-  const response = await fetch('/api/contact', {
-    method: 'POST',
-    headers: { 'Content-Type': 'application/json' },
-    body: JSON.stringify(payload),
-  })
-
-  const result = (await response.json().catch(() => null)) as {
-    error?: string
-    message?: string
-  } | null
-
-  if (!response.ok) {
-    throw new Error(result?.error || 'Failed to send message')
-  }
-
-  return result
+async function submitContactRequest(payload: SubmitContactBody) {
+  return requestJson(submitContactContract, { body: payload })
 }
 
 export function ContactForm() {

apps/sim/app/(landing)/components/demo-request/demo-request-modal.tsx

@@ -14,10 +14,12 @@ import {
   Textarea,
 } from '@/components/emcn'
 import { Check } from '@/components/emcn/icons'
+import { requestJson } from '@/lib/api/client/request'
 import {
   DEMO_REQUEST_COMPANY_SIZE_OPTIONS,
   type DemoRequestPayload,
   demoRequestSchema,
+  submitDemoRequestContract,
 } from '@/lib/api/contracts/demo-requests'
 import { flattenFieldErrors } from '@/lib/api/contracts/primitives'
 import { captureClientEvent } from '@/lib/posthog/client'
@@ -56,22 +58,7 @@ const LANDING_INPUT =
   'h-[32px] rounded-[5px] border border-[var(--border-1)] bg-[var(--surface-5)] px-2.5 font-[430] font-season text-[13.5px] text-[var(--text-primary)] transition-colors placeholder:text-[var(--text-muted)] outline-none'
 
 async function submitDemoRequest(payload: DemoRequestPayload) {
-  const response = await fetch('/api/demo-requests', {
-    method: 'POST',
-    headers: { 'Content-Type': 'application/json' },
-    body: JSON.stringify(payload),
-  })
-
-  const result = (await response.json().catch(() => null)) as {
-    error?: string
-    message?: string
-  } | null
-
-  if (!response.ok) {
-    throw new Error(result?.error || 'Failed to submit demo request')
-  }
-
-  return result
+  return requestJson(submitDemoRequestContract, { body: payload })
 }
 
 export function DemoRequestModal({ children, theme = 'dark' }: DemoRequestModalProps) {

apps/sim/app/(landing)/integrations/components/request-integration-modal.tsx

@@ -12,6 +12,8 @@ import {
   ModalHeader,
   Textarea,
 } from '@/components/emcn'
+import { requestJson } from '@/lib/api/client/request'
+import { integrationRequestContract } from '@/lib/api/contracts/common'
 
 type SubmitStatus = 'idle' | 'submitting' | 'success' | 'error'
 
@@ -46,18 +48,14 @@ export function RequestIntegrationModal() {
       setStatus('submitting')
 
       try {
-        const res = await fetch('/api/help/integration-request', {
-          method: 'POST',
-          headers: { 'Content-Type': 'application/json' },
-          body: JSON.stringify({
+        await requestJson(integrationRequestContract, {
+          body: {
             integrationName: integrationName.trim(),
             email: email.trim(),
             useCase: useCase.trim() || undefined,
-          }),
+          },
         })
 
-        if (!res.ok) throw new Error('Request failed')
-
         setStatus('success')
         setTimeout(() => setOpen(false), 1500)
       } catch {

apps/sim/app/_shell/providers/session-provider.tsx

@@ -4,6 +4,8 @@ import type React from 'react'
 import { createContext, useCallback, useEffect, useMemo, useState } from 'react'
 import { createLogger } from '@sim/logger'
 import { useQueryClient } from '@tanstack/react-query'
+import { requestJson } from '@/lib/api/client/request'
+import { listCreatorOrganizationsContract } from '@/lib/api/contracts/creator-profile'
 import { client } from '@/lib/auth/auth-client'
 import { extractSessionDataFromAuthClientResult } from '@/lib/auth/session-response'
 
@@ -92,14 +94,9 @@ export function SessionProvider({ children }: { children: React.ReactNode }) {
       }
 
       try {
-        const response = await fetch('/api/organizations')
-        if (!response.ok) {
-          return
-        }
+        const orgData = await requestJson(listCreatorOrganizationsContract, {}).catch(() => null)
+        if (!orgData) return
 
-        const orgData = (await response.json()) as {
-          organizations?: Array<{ id: string }>
-        }
         const organizationId = orgData.organizations?.[0]?.id
 
         if (!organizationId || isCancelled) {