feat: Add Anamnesis provider for claude-mem memory systems

[gene-jelly]

Summary

Adds a new anamnesis provider that benchmarks claude-mem memory systems — the observation-based memory layer used by Claude Code.

• Ingest: Transforms benchmark conversations into claude-mem observations (raw text or LLM-extracted via Claude CLI)
• Index: Embeds observations into ChromaDB via direct Python script (avoids version mismatches with chroma-mcp)
• Search: Hybrid semantic (ChromaDB vectors) + keyword (SQLite LIKE) search with namespace isolation
• Clear: Cleans both SQLite observations and ChromaDB embeddings
• CLI model support: Added opus-cli, sonnet-cli, haiku-cli model aliases for answering via Claude CLI subprocess

Benchmark Results (LoCoMo)

Answering Model	Judge	Accuracy	Hit@10	MRR
GPT-4o (conservative prompt)	GPT-4o	40% (8/20)	85%	0.515
GPT-4o (assertive prompt)	GPT-4o	70% (14/20)	90%	0.515
GPT-4.1-mini	GPT-4.1-mini	70% (14/20)	100%	0.612
GPT-5.2	GPT-4.1-mini	80% (16/20)	100%	0.594
GPT-5.2	GPT-5.2	75% (15/20)	100%	0.594

By question type (GPT-5.2 / gpt-4.1-mini judge):

• Single-hop: 87.5% (7/8)
• Multi-hop: 80% (8/10)
• Temporal: 50% (1/2)

Evaluation uses binary LLM-as-Judge scoring (stricter than token F1). Comparable methodology to Mem0's published LoCoMo results (66.9%).

Architecture

LoCoMo conversations → SQLite observations → ChromaDB embeddings
                                                    ↓
                        Search query → ChromaDB semantic + SQLite keyword → RRF-style merge

Key design decisions

• Direct ChromaDB access via Python scripts (embed.py, search.py) instead of HTTP API calls, since claude-mem's worker doesn't expose an embedding endpoint
• CHROMA_PYTHON env var lets users point to the same Python environment as their chroma-mcp process (important for ChromaDB version compatibility)
• Namespace isolation: Each benchmark question gets its own namespace, preventing cross-contamination between questions and with the user's real observations
• Extraction mode (ANAMNESIS_EXTRACTION=true): Uses Claude CLI in print mode to extract structured observations, matching how claude-mem processes conversations in production
• Assertive answer prompt: Targeted instructions for complete extraction, temporal date conversion, and counterfactual reasoning

Environment variables

Variable	Default	Description
ANAMNESIS_WORKER_URL	http://localhost:37777	claude-mem worker URL
ANAMNESIS_DB	~/.claude-mem/claude-mem.db	SQLite database path
ANAMNESIS_EXTRACTION	false	Enable LLM extraction mode
CHROMA_PYTHON	python3	Python with matching ChromaDB version
CHROMA_PATH	~/.claude-mem/vector-db	ChromaDB persistent storage path

Files

• src/providers/anamnesis/index.ts — Provider implementation (612 lines)
• src/providers/anamnesis/prompts.ts — Custom answer prompts optimized for observation format
• src/providers/anamnesis/embed.py — ChromaDB embedding script
• src/providers/anamnesis/search.py — ChromaDB semantic search script
• src/types/provider.ts — Added "anamnesis" to ProviderName
• src/providers/index.ts — Registered AnamnesisProvider
• src/utils/config.ts — Added anamnesis config
• src/utils/models.ts — Added CLI model aliases + GPT-5.2 config
• src/orchestrator/phases/answer.ts — CLI subprocess support + 10min timeout

Test plan

• 20-question LoCoMo run: 80% accuracy (GPT-5.2), 100% Hit@10
• Prompt iteration: 40% → 70% → 80% through targeted prompt engineering
• Multi-model comparison: GPT-4o, GPT-4.1-mini, GPT-5.2
• Multi-judge comparison: gpt-4.1-mini vs gpt-5.2 (95% agreement)
• Embedding verified: all documents embedded successfully
• Hybrid search verified: semantic + keyword fallback working
• Full 50-question LoCoMo run (in progress)

🤖 Generated with Claude Code

[sentry]

Bug: The clear() method's SQL query uses OR project = 'memorybench', which will cause all benchmark data to be deleted, not just data for the specified namespace.
_{Severity: HIGH}

Suggested Fix

Remove the OR project = 'memorybench' condition from the DELETE statement in the clear() method. The query should only use WHERE namespace = ? to ensure data deletion is isolated to the specified container.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: src/providers/anamnesis/index.ts#L578-L590

Potential issue: The `clear()` method in the `anamnesis` provider uses the SQL condition
`WHERE namespace = ? OR project = 'memorybench'`. Since all benchmark observations are
stored with `project = 'memorybench'`, invoking this method for any container will
delete not only that container's data but all observations from all containers. While
the method is not currently called in the main execution flow, its existence poses a
significant risk of accidental mass data deletion if used by future code or external
cleanup scripts.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

[sentry]

Bug: The use of get_collection() can raise a ValueError if the ChromaDB collection is missing, causing semantic search to fail silently and fall back to keyword search.
_{Severity: MEDIUM}

Suggested Fix

Replace the call to client.get_collection(COLLECTION) with client.get_or_create_collection(COLLECTION). This ensures the collection is created if it does not already exist, preventing the ValueError and ensuring semantic search functionality is robust.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: src/providers/anamnesis/embed.py#L44

Potential issue: The code calls `client.get_collection(COLLECTION)` in `embed.py` and
`search.py`, which will raise a `ValueError` if the ChromaDB collection does not exist.
This can occur in scenarios like a fresh installation or a corrupted state. The current
error handling catches this exception but leads to silent failures. In the embedding
phase, the error is logged as a warning, and embeddings are skipped. In the search
phase, it falls back to a keyword-only search. This results in a silent degradation of
the semantic search functionality, potentially impacting benchmark accuracy without
clear indication to the user.

[sentry]

Bug: The semantic search script can crash when requesting more results from ChromaDB than are available, causing a silent fallback to a less accurate keyword-only search.
_{Severity: HIGH}

Suggested Fix

In search.py, wrap the ChromaDB query in a try-except block to handle the NotEnoughElementsException. Alternatively, before querying, determine the number of available documents and request min(limit * 3, num_available_documents).

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: src/providers/anamnesis/search.py#L28-L33

Potential issue: The `search.py` script requests a fixed multiple of the search limit
(`limit * 3`, which is 57) from ChromaDB. However, ChromaDB can raise a
`NotEnoughElementsException` if the number of available documents in the filtered
namespace is less than the requested amount. This is a common scenario as each question
has its own namespace with a relatively small number of documents. The unhandled
exception causes the script to crash. The calling process catches this failure and
silently falls back to a keyword-only search, significantly degrading search quality
without any indication to the user or developer.