feat(cli): forge try — instant-gratification onboarding (#350)#353
feat(cli): forge try — instant-gratification onboarding (#350)#353initializ-mk wants to merge 9 commits into
Conversation
…APIs (#349) The bundled weather skill allowed egress only to api.openweathermap.org and api.weatherapi.com — both require an API key — yet declared no key env at all, so it was dead on arrival. Its tool bodies were also prose only, with no runnable command. Repoint it at wttr.in (free, keyless, accepts a city name directly so it maps 1:1 to the location input) and ship two runnable scripts: - scripts/weather-current.sh '{"location":"Tokyo"}' - scripts/weather-forecast.sh '{"location":"Tokyo","days":3}' Both use curl + jq, need no API key, validate input, check the HTTP status, and emit shaped JSON. egress_domains is now [wttr.in]. Kept in sync: the egress-wizard domain hints (egress_step.go), the scanner test fixture, and the weather example in .claude/skills/forge.md. Verified live against wttr.in (current + forecast); scripts are shellcheck-clean; forge-skills and forge-cli/internal/tui suites green.
…ffold (#350) New `forge try` command that scaffolds a keyless demo agent (the native forge LLM executor + weather skill + http_request/datetime_now/math_calculate builtins) and prints its summary. Ephemeral temp workspace by default, cleaned on exit; --keep writes ./forge-quickstart. Reuses scaffold() via two additions to initOptions (no duplication): - OutputDir overrides the target dir (temp dir / ./forge-quickstart) - Preset stops scaffold() before its banner + auto-run of `forge run`, so try drives the agent in-process (Phases 2-4). Egress auto-derives from the weather skill (allowlist, wttr.in — keyless; depends on #351). No channels, no secrets on disk. No LLM call yet. Verified: ephemeral scaffold + summary + cleanup (no cwd leak); --keep writes a valid forge.yaml; init suite green; gofmt/vet/lint clean.
resolveTryProvider layers an ordering policy over the runner's existing resolution: explicit flags -> env key (ANTHROPIC > OPENAI > GEMINI) -> saved OpenAI OAuth -> local Ollama (short-dial probe) -> interactive picker (TTY) -> actionable error (no TTY). Prints a human 'Using ...' label. Picker offers OpenAI sign-in (runOAuthFlow), paste-key (masked, in-memory), or Ollama. No new credential store; SilenceUsage so the no-credential error stays clean. Verified: env-key path silent + correct label with the documented precedence; no-creds/no-TTY yields the actionable error and exit 1; explicit flags still win. gofmt/vet/lint clean.
…350) New forge-cli/runtime/LocalSession assembles the SAME coreruntime.LLMExecutor that forge run uses (builtin tools + vendored skills, egress-enforced client + subprocess proxy, audit + progress hooks, provider client) WITHOUT an HTTP server, scheduler, MCP, or long-term memory. No second executor — a trimmed bootstrap around the shared sub-builders. History rides in task.History; the executor Store is nil so nothing persists. try.go drives it: RunTurn per line, streaming the reply under 'agent ›', /exit + Ctrl-D + Ctrl-C (signal.NotifyContext) exit, --once single turn, an optional positional prompt seeds the first turn. The runner's JSON logger is silenced so stdout stays clean for the chat. Verified: real turn via a mock OpenAI server (llm_call fires, reply returned, history accumulates to 2 then 4 across turns); session construction (egress proxy + skill registration + client build) succeeds live; runtime + cmd suites green; gofmt/vet/lint clean.
New forge-cli/internal/tryview.Renderer implements the audit Sink contract and
maps the agent's own audit events to inline loop lines: tool calls
(▸ tool name(args)), egress checks (▸ egress domain ✓/✗), guardrail blocks,
and tool results (◂ preview), plus a dim compact 'audit {…}' summary after
each reply. lipgloss orange accent, degrades to plain text with NO_COLOR /
non-TTY. --quiet hides the loop; --audit echoes full NDJSON.
LocalSession now builds its audit logger on a discard base (no stderr noise)
and captures tool args/results (redacted) so the renderer can preview them;
try.go attaches the renderer as an extra sink. Reads existing events only —
no new audit constants, no forge-core TTY code.
Verified: 12 renderer unit tests (event→line mapping, truncation, quiet,
--audit echo, no-color degradation, summary reset) + an end-to-end
LocalSession test rendering a real datetime_now tool call. Suites green;
gofmt/lint clean.
Add the intro banner, three starter prompts (all within the keyless egress allowlist so none dead-ends on a blocked domain), and an exit graduation message that points the user up the ladder: keep the demo (--keep writes ./forge-quickstart), run it with forge serve, or deploy with forge package. Orange accent (lipgloss) on command tokens only; degrades to plain text on non-TTY / NO_COLOR. No em-dashes in printed strings (house style). Verified visually: banner + summary + suggestions render for the interactive REPL; graduation prints on exit (keep-aware); --once stays terse. Build + lint clean.
cmd/try_test.go: quickstartPreset scaffolds a valid, keyless forge.yaml (egress wttr.in, never the key-gated hosts); resolveTryProvider ordering (flags win, ANTHROPIC>OPENAI env precedence, Ollama probe via a live listener, no-creds/no-TTY error with HOME isolated so a dev's real OAuth token can't leak in); tryWorkspace lifecycle (--keep targets cwd with no cleanup; ephemeral temp base is removed). Complements local_session_test.go (real turn + tool-loop render) and the tryview renderer suite. All green; vet + lint clean.
…production (#350) Rewrite quick-start.md around a single command (brew install + forge try), the hero transcript, and a 4-rung ladder (try -> keep+skill -> serve+guardrail -> package). Channels/build/deploy are gone from the quick start. New ship-to-production.md carries the full init -> skills -> secrets -> validate -> run -> build -> package pipeline. README Quick Start leads with forge try + transcript and links both pages; forge.md §15 gets a one-line forge try pointer at the top. Verified: broken-link check clean; no channel steps remain in quick-start.
initializ-mk
left a comment
There was a problem hiding this comment.
Review: forge try — instant-gratification onboarding
Reviewed the seven forge try commits (skipped the leading #351 weather commit — already reviewed on #351; it dedupes on rebase as noted). Strong, well-architected PR; the security-critical claims hold up under tracing. Merge-ready — findings are one cross-PR merge-coordination item and four low/nit polish items.
What I verified holds (the parts that matter)
- Genuinely reuses the shared executor, not a fork.
LocalSessionassembles the samecoreruntime.LLMExecutorvia the runner's sub-builders;Storeis nil so history stays in memory and nothing persists. The "no second executor,Run()untouched" claim is accurate. - Egress is actually wired, not just constructed.
buildTryEgressbuilds both the in-process enforced client (installed on ctx viaWithEgressClientinRunTurn) and the subprocess proxy, both emittingegress_allowed/egress_blocked. This is exactly the "works in isolation vs. where it's mounted" trap this line of work keeps hitting — avoided here. - The paste-key secret doesn't leak into skill subprocesses. I chased this:
try.godoesos.Setenvon the pasted key, butSkillCommandExecutor.Runbuilds a minimal allowlisted env (clean slice,PATH+HOME, then only the skill's declared env vars + proxy/trace/OTel). The weather skill declaresenv.required: [], so it gets neither the key nor the ambient env. No leak in the shipped demo. - Fail-closed where it counts: no credential + no TTY → actionable error + exit 1; ephemeral temp dir removed via
defer RemoveAll(base); no secrets on disk for the ephemeral run. CI fully green.
Findings (see inline)
- Medium (cross-PR merge coordination):
buildTryEgressis a fourth copy of the egress-construction pattern, on the pre-#348 signature — compiles today only because #348 isn't merged; when #348 lands this won't compile until updated, and until thenforge trysilently ignoresallowed_private_cidrs. - Low: egress-resolve failure falls back to a fully-unenforced
http.DefaultClient(fail-open). - Low/nit: the paste-key comment claims the key is persisted under
--keep, but the code never writes it to disk (the safe behavior) — reconcile comment + graduation text. - Nit:
--yesis a dead flag (parsed, never read). - Nit (UX): Ctrl-C at the
you ›input prompt doesn't exit —signal.NotifyContextcatches SIGINT butscanner.Scan()isn't ctx-aware, so it hangs until Enter/Ctrl-D. Honestly documented (Ctrl-D //exit), but Ctrl-C is a reflex in a hero demo. Consider reading stdin in a goroutine and selecting onctx.Done().
Nice work — the audit-stream-as-first-run-delight framing is a genuinely good idea, and the reuse discipline (one executor, trimmed bootstrap) is exactly right.
| allowPrivateIPs = true | ||
| } | ||
|
|
||
| enforcer := security.NewEgressEnforcer(nil, egressCfg.Mode, egressCfg.AllDomains, allowPrivateIPs) |
There was a problem hiding this comment.
Medium (cross-PR merge coordination). This block (security.Resolve at L205, NewEgressEnforcer here at L224, NewEgressProxy at L240) is a fourth copy of the egress-construction pattern, and it's on the pre-#348 signature — Resolve 5-arg, NewEgressEnforcer 4-arg, NewEgressProxy 2-arg. It compiles today only because #348 isn't merged yet. The moment #348 lands (which adds allowedPrivateCIDRs to all three), this file won't compile until updated, and until then forge try will silently ignore allowed_private_cidrs where the server path honors it. Since you own both PRs: land #348 first and rebase this onto it (threading allowedPrivateCIDRs through buildTryEgress), or track it as a required follow-up. Also directly reinforces #348's "consolidate the copies" note — this is now the copy most likely to diverge, since its comment says it "mirrors Run()'s egress setup" but it's a hand-maintained duplicate.
| ) | ||
| if err != nil { | ||
| r.logger.Warn("egress resolve failed; using unenforced client", map[string]any{"error": err.Error()}) | ||
| return http.DefaultClient, "", noop |
There was a problem hiding this comment.
Low. On a security.Resolve error this returns http.DefaultClient — a fully unenforced client — with only a Warn. That's a fail-open on a security path. Effectively unreachable for the fixed quickstart preset (its scaffolded egress config resolves fine), but the wrong default direction: a future config change that makes Resolve fail would silently drop egress enforcement entirely. Prefer a deny-all / empty-allowlist enforced client here so the failure mode stays closed.
| } | ||
|
|
||
| // pasteKeyResolution prompts for a provider and a masked API key, held only in | ||
| // memory (written to disk solely under --keep, via the scaffold env path). |
There was a problem hiding this comment.
Low/nit — comment vs. code. This says the pasted key is "written to disk solely under --keep, via the scaffold env path," but it isn't: the key goes into res.EnvOverrides (session-only, consumed by NewLocalSession), while opts.EnvVars is always map[string]string{} — so the key is never persisted, even under --keep. The never-persist behavior is the correct, safe one and matches the PR's "no secrets on disk" promise, so just fix the comment. Worth surfacing the consequence in the graduation text too: forge try --keep + a pasted key writes ./forge-quickstart with no credential, so a later forge serve there needs the env var set by hand.
| f.onceSet = cmd.Flags().Changed("once") | ||
| f.quiet, _ = cmd.Flags().GetBool("quiet") | ||
| f.audit, _ = cmd.Flags().GetBool("audit") | ||
| f.yes, _ = cmd.Flags().GetBool("yes") |
There was a problem hiding this comment.
Nit — dead flag. f.yes is parsed here but never read anywhere in runTry/downstream. It's advertised in --help ("assume yes to prompts") but does nothing. Either wire it (e.g. auto-select the picker's recommended default so --yes is truly non-interactive) or drop the flag.
…, dead flag) - [Low] egress-resolve failure now fails CLOSED: a deny-all allowlist enforcer instead of an unenforced http.DefaultClient, so a future config that makes Resolve error can't silently drop egress enforcement. - [Nit] drop the dead --yes flag (parsed, never read). - [Low/nit] correct the paste-key comment: the key is session-only and never written to disk, even under --keep; note the consequence (a kept agent has no credential on disk). - [Nit/UX] Ctrl-C at the 'you ›' prompt now exits: stdin reads on a goroutine and the REPL selects on ctx.Done() (scanner.Scan isn't ctx-aware). - [Medium/#348 coupling] add an explicit NOTE that buildTryEgress is on the pre-#348 security signatures and must thread allowedPrivateCIDRs when #348 lands (tracked; can't rebase onto an unmerged branch). Build/vet/lint clean; cmd + runtime + tryview suites green.
|
Addressed the review in a16dc14:
Build/vet/lint clean; cmd + runtime + tryview suites green. Thanks for the careful trace on the egress-mount and paste-key-leak paths. |
initializ-mk
left a comment
There was a problem hiding this comment.
Re-review: fix commit a16dc140 — all actionable findings resolved
Verified each finding against the fix diff. CI fully green (all 6 builds, Lint, Test, Integration, Doc-link).
Finding 2 (Low — fail-open egress) — ✅ resolved
On security.Resolve error, buildTryEgress now returns a deny-all enforced client (NewEgressEnforcer(nil, ModeAllowlist, nil, allowPrivateIPs) — allowlist mode with no domains blocks everything) instead of the unenforced http.DefaultClient. Fail-closed, exactly as recommended. The allowPrivateIPs computation was correctly hoisted above the error branch so both paths share it.
Finding 4 (dead --yes) — ✅ resolved
Removed from the struct, init(), and parseTryFlags — no longer advertised in --help.
Finding 3 (paste-key comment) — ✅ resolved
Comment rewritten accurately: the key is session-only (res.EnvOverrides, consumed by NewLocalSession), never written to disk even under --keep, and it now spells out the consequence (a kept agent has no on-disk credential; a later forge serve needs the env var set by hand).
Finding 5 (Ctrl-C at the you › prompt) — ✅ resolved, correctly
stdin now reads on a goroutine feeding a lines channel; the REPL selects on ctx.Done() (Ctrl-C → newline + return), eof (Ctrl-D → return), or a line. The one-read goroutine leak on exit is real but harmless (the process is exiting) and honestly acknowledged in the comment. Mid-turn cancellation and graduation-on-exit both still work.
Finding 1 (#348 coupling) — appropriately deferred with an explicit NOTE
You can't rebase onto an unmerged branch, so the right move is what was done: a prominent in-code NOTE on buildTryEgress documenting that the three constructors are on pre-#348 signatures and must thread allowedPrivateCIDRs when #348 lands, cross-referencing #348's consolidation note. Keeps the landmine visible for whoever merges #348 second.
Verdict: all fixes correct, nothing dangling. Merge-ready (modulo the #348 sequencing the NOTE now tracks).
One optional residual, non-blocking and on an effectively-unreachable branch (the fixed preset always resolves): on the resolve-error path the in-process client is now fail-closed, but it still returns proxyURL="", so skill subprocesses would run unproxied rather than blocked. Fully fail-closing that dead branch would mean refusing to start when egress can't resolve — reasonable to leave as-is behind the NOTE.
Implements #350: a single
forge trycommand that takes a first-time user from install to talking to a working agent whose loop they can watch, in under 60 seconds, with zero required decisions.What it does
The differentiator: the agent's own audit stream (tool calls, egress checks, guardrail blocks) renders inline — the same signal the enterprise story rests on, surfaced as first-run delight.
Design (per the plan, locked)
runtime.LocalSessionassembles the samecoreruntime.LLMExecutorforge runuses (builtin tools + vendored skills, egress-enforced client + subprocess proxy, audit + progress hooks, provider client) minus the server/scheduler/MCP/memory. No second executor — a trimmed bootstrap around the shared sub-builders;Run()is untouched (zero risk to the server path).--keepwrites./forge-quickstart. No secrets on disk for the ephemeral run.scaffold()via twoinitOptionsfields (OutputDir,Preset) — no forked init logic.forge-cli, degrades to plain text (NO_COLOR / non-TTY).--quiethides it,--auditshows full NDJSON.Phases (one commit each)
forge try; pipeline moved to newship-to-production.mdVerification
LocalSessionmakes the LLM call, history accumulates); end-to-end test renders a realdatetime_nowtool call.--auditecho, no-color);resolveTryProviderordering + workspace-lifecycle tests.go build ./...,go vet ./...,go test ./forge-cli/...all green;gofmt+golangci-lintclean; docs broken-link check clean; no channels in the quick start.Out of scope (named, not built)
Hosted metered demo key,
forge try --ui, a second demo agent — all deferred per the plan.