feat: MLflow tracing with async Stop hook (opt-in)

[dgokeeffe]

Priority

P1 — silent feature failure. README claims "every Claude Code session is automatically traced — zero configuration required", but the deployed setup_mlflow.py hardcodes MLFLOW_CLAUDE_TRACING_ENABLED="false", which makes the upstream Stop hook short-circuit before any trace is written. No traces, no error, no breadcrumb. Every user thinks tracing works; nothing does. This PR introduces the opt-in env-var override so users who want it can flip it on.

---

Summary

Migrates the work from datasciencemonkey PR #139 to the new repo home, rebased onto databrickslabs/main.

• Opt-in tracing via MLFLOW_CLAUDE_TRACING_ENABLED=true in app.yaml — keeps default behaviour unchanged for existing deployments, but gives users a single env-var to flip.
• Stop hook delegates to mlflow-trace-stop.sh, which backgrounds the handler via nohup timeout 30 … & disown. Returns in <1s so the rest of the Stop hook chain isn't blocked.
• Hook-event JSON via temp file captured synchronously before backgrounding — naive nohup would redirect stdin to /dev/null and the handler would lose the transcript path.
• Hard 30s ceiling on the backgrounded flush prevents a wedged handler from leaking memory/CPU.
• Pins mlflow-skinny and mlflow-tracing to 3.11.1 to match the Apps runtime — version mismatches caused silent import failures.

Why this matters

Issue #9 documents that today's MLFLOW_CLAUDE_TRACING_ENABLED="false" in setup_mlflow.py causes the upstream Stop hook to short-circuit before any trace is written. The README promises auto-tracing; the code delivers nothing. This PR introduces a real opt-in path so the experiment actually gets populated when a user wants it on.

Test plan

• uv run pytest tests/test_mlflow_tracing.py — should pass locally
• Deploy with MLFLOW_CLAUDE_TRACING_ENABLED=true, run a session, confirm trace appears at /Users/{owner}/{app_name} in MLflow
• Verify Stop hook returns in <2s (doesn't block session teardown)
• Verify that without the env var, no hook is registered in ~/.claude/settings.json (default-off behaviour preserved)

Closes #9
Closes #11

This pull request and its description were written by Isaac.

---

Test Evidence (verified on the live deployment 2026-05-06)

Triple-confirmed the bug on a fresh databricks/main v0.18.1 deployment:

1. Settings on disk (read directly from the running container):

$ python3 -c "import json; s=json.load(open('/app/python/source_code/.claude/settings.json')); \
              print('FLAG=', s['env']['MLFLOW_CLAUDE_TRACING_ENABLED'], \
                    'EXP=', s['env']['MLFLOW_EXPERIMENT_NAME'])"
FLAG= false  EXP= /Users/<owner>/coding-agents

2. Upstream Stop hook gates on the flag (mlflow-skinny 3.11.1):

# mlflow/claude_code/hooks.py:199
def stop_hook_handler() -> None:
    if not is_tracing_enabled():
        response = get_hook_response()
        print(json.dumps(response))
        return    # <-- exits before processing transcript

3. MLflow experiment doesn't even exist on the deployed workspace:

$ curl -H "Authorization: Bearer $PAT" \
    "$HOST/api/2.0/mlflow/experiments/get-by-name?experiment_name=/Users/<owner>/coding-agents"
{"error_code": "RESOURCE_DOES_NOT_EXIST",
 "message": "Node /Users/<owner>/coding-agents does not exist."}

So nothing is written. MLflow auto-creates experiments on first write — the Stop hook short-circuits before any write, so the experiment never materializes and the user has no trail to follow when they go looking.

Adjacent finding: tests/test_mlflow_tracing.py:63 is named test_tracing_enabled but asserts ... == "false" — a green test certifying the broken state. Fix should rename the assertion to match the new behaviour.

[dgokeeffe]

@datasciencemonkey — flagging for review. P1: README claims auto-tracing works on every session; deployed code has the master flag hardcoded "false" so the upstream Stop hook short-circuits silently. Triple-verified on a live v0.18.1 deploy (settings.json, upstream hook source, MLflow REST 404 on the experiment). Evidence in PR body.