structured-extractor

A small command-line tool that extracts clean, structured JSON from messy, unstructured text — a sales email, a shipping notice, a meeting note, an invoice — using the Anthropic API (Claude).

It pulls out a defined set of fields (sender, recipient, date, line items with quantities and amounts, totals, deadlines, reference numbers), validates the model's output against a typed schema, and repairs/retries when the model returns something malformed instead of crashing.

What it does

1. Reads free text from a file argument or from stdin.
2. Sends it to Claude with a clear, schema-describing prompt.
3. Extracts the JSON object from the response (tolerating code fences or prose around it).
4. Validates it against a Zod schema.
5. On any failure — invalid JSON, missing fields, wrong types — feeds the error back to the model and tries again (default 3 attempts). If it still can't get valid data, it exits with a clear error rather than emitting garbage.

Requirements

• Node.js 18+
• An Anthropic API key in the ANTHROPIC_API_KEY environment variable (never hardcoded).

Install

npm install

Configure

export ANTHROPIC_API_KEY=sk-ant-...

Run

Without building, via tsx:

# From a file
npm run extract -- examples/sales-email.txt

# From stdin
cat examples/sales-email.txt | npm run extract

Or build once and use the extract binary:

npm run build
node dist/cli.js examples/sales-email.txt

Example

Input (examples/sales-email.txt):

From: Dana Whitfield <dana@brightpack.co>
To: procurement@acmewidgets.com
Subject: Re: Q3 reorder

Hi team,

Following up on our call — here's the reorder for the warehouse:

  - 500 x Standard Mailer Box (BX-200) at $1.20 each
  - 120 x Bubble Wrap Roll (BW-50), $14.50 a roll
  - 12 x Packing Tape (24-pack) — 38.00 per pack

That comes to about $4,386 all in (USD). Our PO number on this is PO-99821.

Could you confirm by Friday, 27 June? We'd need the shipment to arrive no later
than July 11 to make our restock window.

Thanks,
Dana

Output:

{
  "documentType": "sales_email",
  "sender": "Dana Whitfield <dana@brightpack.co>",
  "recipient": "procurement@acmewidgets.com",
  "date": null,
  "reference": "PO-99821",
  "currency": "USD",
  "items": [
    { "description": "Standard Mailer Box (BX-200)", "quantity": 500, "unitAmount": 1.2, "totalAmount": 600 },
    { "description": "Bubble Wrap Roll (BW-50)", "quantity": 120, "unitAmount": 14.5, "totalAmount": 1740 },
    { "description": "Packing Tape (24-pack)", "quantity": 12, "unitAmount": 38, "totalAmount": 456 }
  ],
  "totalAmount": 4386,
  "deadlines": [
    { "description": "Confirm reorder", "date": "2024-06-27" },
    { "description": "Shipment must arrive", "date": "2024-07-11" }
  ],
  "notes": null
}

(The exact values the model returns may vary slightly — e.g. how it phrases a deadline description, or whether it resolves a year-less date.)

Tests

npm test

The tests use a mocked model caller, so they run without a live API key and are fully deterministic. They cover:

• Schema validation (tests/schema.test.ts) — valid objects pass; missing required lists, wrong types, and incomplete line items are rejected.
• JSON repair (tests/repair.test.ts) — extracting the object from clean output, ```json fences, bare fences, and surrounding prose; rejecting input with no JSON or invalid syntax.
• The extract/retry loop (tests/extract.test.ts) — succeeds on a good first response, recovers after an unparseable or schema-invalid response, and throws after exhausting all attempts.

Project layout

File	Responsibility
src/schema.ts	Zod schema + inferred TypeScript types — the single source of truth for the output shape.
src/prompt.ts	All prompt text, isolated for easy reading/tweaking.
src/repair.ts	Pure helpers that recover JSON from messy model output.
src/extract.ts	Orchestration: call → parse → validate → repair-retry. Depends only on an injectable ModelCaller.
src/client.ts	Thin Anthropic API wiring; reads the key from the environment.
src/cli.ts	File/stdin input handling and output.

Design notes

• Schema as the contract. The Zod object both validates at runtime and generates the static types via z.infer, so there's no separate type definition to drift out of sync.

• The model caller is an interface, not a hard dependency. extract.ts depends on ModelCaller = (messages) => Promise<string>, and client.ts provides the real, API-backed implementation. That seam is what lets the tests exercise the entire repair loop with a scripted fake and no network.

• Repair via the model, not a JSON fixer. Rather than hand-rolling fixes for trailing commas and the like, malformed output is sent back to the model with the specific error. The model is far better at correcting its own output than a brittle string patcher would be. The pure extraction helpers only handle the common, safe cases (fences, surrounding prose).

• Why not the API's structured-output mode? Claude can constrain responses to a JSON Schema, which would make most responses valid by construction. That would be the right hardening step for production — but it would also make the parse/validate/repair handling (the heart of this brief) dead code. Here the robust handling is implemented explicitly; structured outputs would layer on top of it cleanly.

• Thinking left off. Field extraction from a short document is well-scoped and doesn't need extended reasoning, so the request omits it to keep latency and cost down. For long or ambiguous documents, enable adaptive thinking in client.ts.