ChatStudyAbroad → Rya migration: core primitives (Track A) + csa-counsellor example (Track B) by lokesh-danu · Pull Request #1 · plexe-ai/rya

lokesh-danu · 2026-07-23T09:39:18Z

Track A — core Rya runtime primitives (ChatStudyAbroad → Rya migration)

This is the core-runtime half of the CSA-counsellor migration: three cross-cutting guarantees that were prompt conventions in the production ChatStudyAbroad agent, promoted into reusable Rya primitives. The csa-counsellor example that consumes them (Track B) lands in a separate PR.

No behaviour changes for existing agents — every primitive is opt-in (manifest field / rya.guard.yaml block / require_user flag).

A1 — declarative tool-call retry + self-heal repair

ToolDecl.retry (max_attempts / backoff / on), with a YAML-1.1 rescue so an unquoted on: key can't silently become a no-op.
@agent.repair("<tool>") callback: a RyaRecoverableToolError self-heals once with a patched input (tool.repair trace step). Repair (domain) and retry (transient) are orthogonal budgets.
_http_tool now tags a 5xx (http_status) and a socket timeout (E_TIMEOUT) so the retry classifier can see them.

A2 — id-secrecy output guard

secrecy_scrub / secrecy_check in guard.py: rewrites configured secret-id patterns to a safe token (the runtime form of prod's scrubMasterIds()) instead of blocking the turn.
Wired at the tool-result boundary (before the model / journal / trace see it) and on outbound (before bytes leave). Opt in via secrecy.enabled.

A3 — per-user connection credential + reconnect-on-401

ctx.connections.upsert + store.upsert_connection (overwrite-in-place on (provider, owner)) so a re-login never leaves a stale duplicate.
require_user fails closed with E_NO_IDENTITY rather than falling through to a workspace-shared credential.
A 401 on a credentialed request → E_CONNECTION_EXPIRED → a distinct needs_reconnect terminal run status (mirrors prod's CrizacAuthError → reconnect prompt; no auto-refresh).
rya deploy --check blocks E_PLAINTEXT_SECRET_AT_REST when a require_user provider has an unencrypted connection at rest.

A5 — in-turn adoption

Manifest adopt: {field: scope.key} copies a successful result field into scoped memory so a later pinned tool in the same turn adopts it (e.g. create_lead → student_state.camsId).

Note on commit structure

Four commits, ordered so deps land before use (guard → store → runtime → sdk). The tool-call pipeline is genuinely one atomic refactor (retry + scrub + adoption share one indivisible code path, as does the _http_tool 5xx/401 handling), so the cleanly-separable pieces got their own commits and the entangled SDK core landed as the final commit — rather than faking a per-primitive split that would produce broken intermediate states.

Tests

New: tests/test_retry_repair.py (A1), tests/test_connection_a3.py (A3, incl. expired_token_surfaces_reconnect — no offline provider can synthesize an upstream 401, so this is a core test rather than a live eval). Extended: tests/test_guard.py (A2, ported from chatstudyabroad/lib/utils.test.ts). 38 passing.

🤖 Generated with Claude Code

Add a secrecy output guard that rewrites configured secret-id patterns to a safe token instead of blocking the turn — the runtime form of production's scrubMasterIds(). Unlike grounding (which fails closed), secrecy scrubs and lets the redacted value through so the model keeps working. - secrecy_scrub_text / secrecy_scrub (per-leaf over dicts/lists) / secrecy_check - _compile_secrecy with a policy-keyed cache; a malformed pattern is skipped, never fatal, so one broken rule can't take down every tool call - opt-in via `secrecy.enabled` in rya.guard.yaml; applied by callers at the tool boundary and on outbound (wired in a later commit) Tests port chatstudyabroad/lib/utils.test.ts (scrubMasterIds): a Crizac master id (3-8 letters + 8+ digits) is scrubbed; numeric CAMS ids, passports, phones, years, and money are preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add upsert_connection to both FileStore and PostgresStore, keyed on (provider, owner). A login handler that re-mints a per-user bearer must not leave a stale duplicate the runtime could later inject, so the upsert rewrites the existing active connection in place — preserving id/createdAt, stamping updatedAt — rather than appending a new one. - FileStore: scan connections_dir for the existing (provider, owner, active) doc and rewrite its file; else create. - PostgresStore: SELECT ... FOR UPDATE on `owner IS NOT DISTINCT FROM %s`, then UPDATE or INSERT. Backs the ctx.connections.upsert SDK path (wired in a later commit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…oy gate (A3) A per-user connection can expire mid-turn. Surface that as a clean, distinct outcome the caller can act on (log in again and retry) instead of a generic failure — mirroring production's CrizacAuthError -> reconnect prompt. No auto-refresh. - engine: an E_CONNECTION_EXPIRED RyaError sets run.status = needs_reconnect with its own run.needs_reconnect trace step (not run.failed); status added to the observability export gate. - turns: needs_reconnect added to TERMINAL_RUN_STATUSES. - api/app: needs_reconnect added to the manual run-status allow-list. - readiness: `rya deploy --check` now blocks E_PLAINTEXT_SECRET_AT_REST when a require_user provider has a stored connection sitting unencrypted at rest — seal() degrades to plaintext without key material, so this fails closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…edential enforcement (A1/A2/A5/A3) Rewrite the _Tools.call backend path so retry, self-heal, secrecy, and adoption all flow through one journaled step, backend-agnostic across agent/http/mock — so a replay after an approval pause returns the memoized result and never re-runs a retry or repair. A1 — declarative retry + repair: - manifest RetryDecl (max_attempts / backoff / on), with a YAML-1.1 `on: true` rescue so an unquoted `on:` key is never a silent no-op. - @agent.repair("<tool>") callback; a RyaRecoverableToolError self-heals once with a patched input (tool.repair trace step). Repair (domain) and retry (transient) are orthogonal budgets. - _http_tool tags a 5xx (http_status) and a socket timeout (E_TIMEOUT) so the retry classifier can see them; RyaError carries http_status. A2 — secrecy wiring: the scrub runs on every tool result before it re-enters the loop / journal / trace, and on every outbound message before it leaves. ctx.guard.check_secrecy exposed for handler-side assertions. A5 — adoption: a manifest `adopt: {field: scope.key}` copies a successful result field into scoped memory so a later pinned tool in the same turn adopts it (e.g. create_lead -> student_state.camsId). A3 — per-user credential: credential resolution is scoped to egress backends only (a local @agent.tool leaf never receives the secret, so provider: on it is governance metadata); require_user fails closed with E_NO_IDENTITY when no verified sub; _http_tool maps a 401 on a credentialed request to E_CONNECTION_EXPIRED; ctx.connections.upsert mints/refreshes the caller's connection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Source-of-truth spec (PRD.md) plus the living migration plan (MIGRATION.md) that ports the production ChatStudyAbroad counsellor onto Rya in five phases, and MIGRATION_CRITIQUE.md — a claim-by-claim verification of the plan against both trees (prod chatstudyabroad/ and the Rya runtime). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Take the governance shell to functional parity with production ChatStudyAbroad. Previously src/agent.py only called single-turn ctx.llm.respond and none of the 28 tools were wired; now it runs the governed 6-step tool loop and implements the tool handlers as @agent.tool leaves. - src/agent.py: COUNSELLOR_SYSTEM prompt (tool-first, currency discipline, discovery-only-via-course_catalogue, presenting != pinning, master-id rule); discovery / CAMS-lookup / workspace / write handlers; @agent.repair for create_lead's domain self-heal (closest valid destination/course, home-state spelling); payload-driven approval-gated writes (loan/email/update); Plexe offer_prediction with graceful degradation; draft_sop/draft_lor; curated seeds (home countries, visa, scholarships); UI cards via ctx.emit_ui; the per-counsellor Crizac login-mint (upserts the bearer per identity). - rya.agent.yaml: provider/scopes/require_user + live url on the Crizac tools (local handler wins offline; url is the live egress seam), retry blocks, and memory.student_state.camsId pins for adoption. - rya.guard.yaml: id-secrecy scrub block (Crizac master id), Crizac + Plexe egress allow-rules, exfiltration/payment denies. - data/: catalogue + student seed fixtures the offline handlers read. Consumes the Track A core primitives (retry/repair, id-secrecy scrub, per-user credential + reconnect, adoption). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…tests - rya.evals.yaml: live routing/governance checks that hold regardless of the model's wording — discovery routes to course_catalogue and never to the disabled tools, master-id scrub token surfaces, gated writes suspend for a human and are never in the loop, offer_prediction degrades gracefully. - tests/: deterministic gates for the correctness contracts the evals can't prove — the master-id-vs-CAMS-id secrecy contract and cards (phase 2), retry-never-double-creates + repair self-heal + camsId adoption (phase 3), pinned-camsId baking + grounding + gated-write landing (phase 4), tail-leaf units + offer_prediction graceful + login-mint upsert (phase 5). 48 passing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

chat.py — a tiny stdin loop over one session (keyed by --email so learned memory and the pinned student persist across turns), filling the gap left by the single-shot `rya events send`. Walks through any approval-gated write inline, showing the queued action (not the model's prose, since gated tools are never exposed to the model) and firing it only on approval. Handles the needs_reconnect outcome. Uses the real model when ANTHROPIC_API_KEY is set, or RYA_FORCE_MOCK=1 for a deterministic offline run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

lokesh-danu · 2026-07-23T09:47:40Z

Added: Track B — the csa-counsellor example that consumes these primitives

Pushed 4 more commits so this PR now carries both halves of the migration — the core Track A primitives and the ChatStudyAbroad → Rya example that exercises them end-to-end. Track B is examples/csa-counsellor/** only; it touches no core files.

docs(csa-counsellor) — PRD, the 5-phase migration spec, and a claim-by-claim critique of the plan verified against both trees.
example(csa-counsellor): wire the governed agent — takes the governance shell from a single-turn stub to the governed 6-step tool loop with real @agent.tool handlers (discovery / CAMS lookups / workspace / writes), @agent.repair self-heal for create_lead, payload-driven approval-gated writes, Plexe offer_prediction with graceful degradation, the per-counsellor Crizac login-mint, and UI cards. Manifest carries provider/scopes/require_user + live url: on the Crizac tools (local handler wins offline, url is the live egress seam), retry blocks, and memory.student_state.camsId adoption pins. Guard adds the id-secrecy scrub block + Crizac/Plexe egress rules.
example(csa-counsellor): evals + tests — live routing/governance evals plus deterministic per-phase gates for the contracts evals can't prove (master-id-vs-CAMS-id secrecy, retry-never-double-creates, camsId adoption, pinned-write landing, offer_prediction graceful, login-mint upsert).
example(csa-counsellor): interactive dev REPL — chat.py, a one-session stdin loop for driving the agent locally (real model or RYA_FORCE_MOCK=1), walking approval-gated writes inline.

Tests: 86 passing on the branch (38 Track A core + 48 csa-counsellor phase tests).

How the two tracks fit: each Track A primitive (retry/repair, id-secrecy scrub, per-user credential + reconnect, adoption) has csa-counsellor as its first real consumer, so reviewing them together shows the primitive and its usage in one place.

vaibs-d · 2026-07-23T14:18:30Z

+            if e.code == "E_CONNECTION_EXPIRED":
+                # A distinct, non-generic outcome: the connection expired mid-turn,
+                # so the run needs the user to reconnect (log in again) and retry —
+                # not a bug. Mirrors prod's CrizacAuthError → clean reconnect prompt.


Can we avoid writing about Crizac specifically? Might be a good idea to scrub the code base for any customer specific references so we can open source Rya later on

vaibs-d

In fact, we can remove the whole csa-counsellor example altogether. What do you think? We can add some templates later

vaibs-d · 2026-07-23T14:18:53Z

@@ -0,0 +1,71 @@
+{
+  "_note": "Local seed for the Crizac catalogue, so Phase-1 discovery runs offline. Each university carries the (country,intake,courseType) it is offered under plus its eligibility-checked courses. Swap course_catalogue for a `url:` tool (Phase 3) to hit the live Crizac endpoints; the return shapes below mirror mapCrizacUniversityFilter / mapCrizacEligibleCourse.",


Remove crizac references.

vaibs-d · 2026-07-23T14:19:07Z

+    numeric CAMS id and passport untouched. Mirrors utils.test.ts scrubMasterIds."""
+    from rya.guard import _compile_secrecy, secrecy_scrub
+    policy = {"secrecy": {"enabled": True, "patterns": [
+        {"id": "crizac_master_id", "kind": "regex",


Same as above

vaibs-d · 2026-07-23T14:19:33Z

+
+
+def test_collect_profile_reports_missing(m):
+    r = run(m.collect_profile({"camsId": "IjmQ1782803306"}))


Cams Id is very specific to Crizac, this CSA example should be generic

vaibs-d · 2026-07-23T14:20:34Z

+def test_gated_tool_never_in_loop(tmp_path, monkeypatch):
+    monkeypatch.setenv("RYA_FORCE_MOCK", "1")
+    engine, agent, store = _engine(tmp_path)
+    gated = ["cams_update_profile", "crizac_update_application", "submit_scholarship_enquiry",


Crizac ref - remove

vaibs-d · 2026-07-23T14:20:46Z

+* **Tail-leaf units** load a FRESH agent module and call each new leaf directly:
+  the discovery tail (programme_detail, university_search/_recommendations,
+  scholarship_search + dedup, crizac_filters, visa_requirements), offer_prediction
+  (graceful when Plexe is unconfigured — the `offer_prediction_graceful_when_down`
+  contract), and draft_sop/draft_lor (grounded context, no fabrication).
+* **Card unit** proves scholarship_search maps to a `scholarships` frame.
+* **A3 wiring** proves the Crizac tools declare a live connection (provider +
+  require_user + url), the gated Crizac writes stay handler-run (no url), and the
+  login-mint upserts a per-counsellor bearer.


Same as above

vaibs-d · 2026-07-23T14:21:02Z

+# ---- A3 live-connection wiring ---------------------------------------------
+def test_crizac_tools_declare_live_connection():
+    manifest = load_manifest(_CSA / "rya.agent.yaml")
+    tools = {t.id: t for t in manifest.tools}
+    # read/create CRM tools: provider + require_user + a live url (handler wins offline)
+    for tid in ["cams_lookup_student", "duplicates_check", "application_activity",
+                "course_catalogue", "programme_detail", "crizac_filters",
+                "create_lead", "create_leads_bulk"]:
+        t = tools[tid]
+        assert t.provider == "crizac" and t.require_user is True and t.url, tid
+    # gated Crizac writes: provider for governance, but NO url (approved action runs
+    # the local handler via engine._execute_action, which would take a url first).
+    for tid in ["crizac_update_application", "submit_scholarship_enquiry", "cams_update_profile"]:
+        t = tools[tid]
+        assert t.provider == "crizac" and not t.url, tid
+
+
+def test_extract_crizac_token_order(m):
+    assert m._extract_crizac_token({"data": {"token": "T"}}) == "T"
+    assert m._extract_crizac_token({"data": {"accessToken": "A"}}) == "A"
+    assert m._extract_crizac_token({"token": "X"}) == "X"
+    assert m._extract_crizac_token({"accessToken": "Y"}) == "Y"
+    assert m._extract_crizac_token({"nope": 1}) is None


Same as above

vaibs-d · 2026-07-23T14:21:13Z

@@ -0,0 +1,535 @@
+# ChatStudyAbroad — Crizac Agent Console


vaibs-d · 2026-07-23T14:21:23Z

@@ -0,0 +1,375 @@
+# Critique of `MIGRATION.md` (ChatStudyAbroad → Rya / csa-counsellor)


vaibs-d · 2026-07-23T14:21:30Z

@@ -0,0 +1,519 @@
+# Migration plan: ChatStudyAbroad → Rya (csa-counsellor)


vaibs-d · 2026-07-23T14:21:51Z

+@pytest.mark.parametrize("raw,want", [
+    # IS scrubbed — 3-8 leading letters + 8+ digits = a Crizac master id.
+    ("The master id is IjmQ1782803306 for this student.",
+     "The master id is (id hidden) for this student."),
+    ("YKHw1782723298 / sGog1782764730", "(id hidden) / (id hidden)"),
+    # NOT scrubbed — the production negatives (utils.test.ts:249-257).
+    ("CAMS 1472802 was created", "CAMS 1472802 was created"),          # numeric CAMS id
+    ("passport MEGHA1234", "passport MEGHA1234"),                       # only 4 trailing digits
+    ("passport Z1234567", "passport Z1234567"),                         # 1 letter + 7 digits
+    ("September 2026, $33,800, 9903105259", "September 2026, $33,800, 9903105259"),  # year/money/phone
+    ("", ""),
+    ("hello", "hello"),
+])


lokesh-danu · 2026-07-23T16:25:19Z

Superseded by #2, which contains ONLY the Track A core-runtime primitives (branch track-a-core). Track B (the csa-counsellor example) is moving to the chatstudyabroad repo. Closing this combined PR.

lokesh-danu and others added 4 commits July 23, 2026 09:38

lokesh-danu requested review from arshadshk and vaibs-d July 23, 2026 09:40

lokesh-danu and others added 4 commits July 23, 2026 09:47

lokesh-danu changed the title ~~Track A: core Rya primitives — tool retry/repair, id-secrecy scrub, per-user credential + reconnect~~ ChatStudyAbroad → Rya migration: core primitives (Track A) + csa-counsellor example (Track B) Jul 23, 2026

vaibs-d reviewed Jul 23, 2026

View reviewed changes

vaibs-d requested changes Jul 23, 2026

View reviewed changes

lokesh-danu mentioned this pull request Jul 23, 2026

Rya core (Track A): retry/repair, credentials, streaming chat, Langfuse datasets #2

Open

lokesh-danu closed this Jul 23, 2026

		@@ -0,0 +1,71 @@
		{
		"_note": "Local seed for the Crizac catalogue, so Phase-1 discovery runs offline. Each university carries the (country,intake,courseType) it is offered under plus its eligibility-checked courses. Swap course_catalogue for a `url:` tool (Phase 3) to hit the live Crizac endpoints; the return shapes below mirror mapCrizacUniversityFilter / mapCrizacEligibleCourse.",



		def test_collect_profile_reports_missing(m):
		r = run(m.collect_profile({"camsId": "IjmQ1782803306"}))

		@@ -0,0 +1,375 @@
		# Critique of `MIGRATION.md` (ChatStudyAbroad → Rya / csa-counsellor)

		@@ -0,0 +1,519 @@
		# Migration plan: ChatStudyAbroad → Rya (csa-counsellor)

Uh oh!

Conversation

lokesh-danu commented Jul 23, 2026

Track A — core Rya runtime primitives (ChatStudyAbroad → Rya migration)

A1 — declarative tool-call retry + self-heal repair

A2 — id-secrecy output guard

A3 — per-user connection credential + reconnect-on-401

A5 — in-turn adoption

Note on commit structure

Tests

Uh oh!

lokesh-danu commented Jul 23, 2026

Added: Track B — the csa-counsellor example that consumes these primitives

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vaibs-d left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lokesh-danu commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants