Skip to content

Commit 4bfecd6

Browse files
committed
docs(rfd): Add RFD 063 — Usage-Based Wizard Field Ordering
Introduces RFD 063, which extends the interactive config wizard (RFD 061) with a frecency-based middle tier in the field selector, powered by the CLI usage tracking data from RFD 062. Fields the user frequently sets via `--cfg` or dedicated CLI flags are promoted in the wizard list, making it increasingly personalized over time. The ranking combines logarithmic frequency scaling with a recency decay (half-life of ~7 days), so stale usage fades naturally without requiring manual resets. The design introduces a three-tier field ordering: already-configured fields first, then frecent fields, then all remaining fields in natural order. Each tier gets a distinct visual indicator (`●`, `◦`, or none). The `interactive_config_browser` function gains a `&CliUsage` parameter, and a `field_usage_scores()` helper translates raw `CliUsage` data into per-field-path scores by parsing `--cfg` values through `KvAssignment` and reverse-mapping dedicated CLI args via the `CliRecord` registry from RFD 060. Signed-off-by: Jean Mertz <git@jeanmertz.com>
1 parent ab523df commit 4bfecd6

2 files changed

Lines changed: 277 additions & 2 deletions

File tree

docs/.vitepress/rfd-summaries.json

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -236,15 +236,19 @@
236236
"summary": "Add `jp completions` and `jp manpage` subcommands using clap_complete and clap_mangen for shell integration."
237237
},
238238
"060-config-explain.md": {
239-
"hash": "9aab2d559dfa612cc7580293424e57a9cd7b0887b1b7766ac5e8dd2a39b123d3",
239+
"hash": "d4f0dcbb9e0c7a0ab02adfd67cd5bf3676c4065e03967ce9bc47740a3a4be9ca",
240240
"summary": "Global `--explain` flag traces config resolution through 9 layers, showing where each field value originates."
241241
},
242242
"061-interactive-config.md": {
243243
"hash": "0b3bcc7ed9bbafde9e3599e05238320856c4e5711af34cc35dbade7d4d0ecbc8",
244244
"summary": "Bare `--cfg` flag triggers interactive configuration browser for searching, inspecting, and editing config fields with type-appropriate prompts."
245245
},
246246
"062-cli-usage-tracking.md": {
247-
"hash": "2a7d38872df128034bd34fa8df7030af0d7f0e97a06cf5d9d522631818805e04",
247+
"hash": "4d3e9ea11f08ea39f327f4e18ca0e65c7e963d936e54953b4917fc507becf274",
248248
"summary": "Adds local-only CLI usage tracking to record command invocations and argument patterns per workspace for adaptive features."
249+
},
250+
"063-usage-based-wizard-field-ordering.md": {
251+
"hash": "0ad593a54402a40cd633d40994157ad8b4958de5dd6d7246b0c6924138fa54b2",
252+
"summary": "Extend config wizard with frecency-based field ordering using CLI usage tracking data."
249253
}
250254
}
Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
# RFD 063: Usage-Based Wizard Field Ordering
2+
3+
- **Status**: Discussion
4+
- **Category**: Design
5+
- **Authors**: Jean Mertz <git@jeanmertz.com>
6+
- **Date**: 2026-03-21
7+
8+
## Summary
9+
10+
Extend the interactive config wizard ([RFD 061]) with a frecency-based tier in
11+
the field selector ordering, powered by the CLI usage tracking infrastructure
12+
from [RFD 062]. Fields the user frequently sets via `--cfg` or dedicated CLI
13+
flags are promoted in the list, making the wizard increasingly personalized over
14+
time.
15+
16+
## Motivation
17+
18+
[RFD 061] ships the interactive config wizard with a two-tier field ordering:
19+
configured (non-default) fields first, then all remaining fields in natural
20+
order. This is functional but static — a user who sets `assistant.tool_choice`
21+
on every other invocation sees it in the same position as a field they've never
22+
touched.
23+
24+
With [RFD 062]'s usage tracking in place, the wizard has access to per-flag
25+
usage data: which `--cfg` fields and dedicated flags are used, how often, and
26+
when. This RFD adds a middle tier — "frecent" (frequently + recently used) —
27+
that promotes fields based on this data.
28+
29+
## Design
30+
31+
### Field ordering (updated)
32+
33+
The wizard's field selector orders fields in three tiers:
34+
35+
1. **Already configured** (non-default) fields — marked with a visual
36+
indicator (e.g., `` prefix).
37+
2. **Frecent** fields — fields the user has set in previous
38+
invocations but that are currently at their default value. Marked with a
39+
distinct indicator (e.g., `` prefix or dimmed text).
40+
3. **All remaining** fields in their natural order (as returned by
41+
`AppConfig::fields()`).
42+
43+
Fields configured during the current wizard session are also marked, distinct
44+
from fields configured by other layers.
45+
46+
### Data sources
47+
48+
The ranking signal comes from two places in the `CliUsage` data ([RFD 062]):
49+
50+
1. **`--cfg` argument values**: The `values` map under
51+
`cli.commands.query.args.config` contains entries like
52+
`assistant.tool_choice=auto` (raw `KEY=VALUE` strings). The clap argument ID
53+
is `config` (the Rust field name in `Globals`), though users know it as
54+
`--cfg` / `-c`. The wizard groups these by field path on the read side —
55+
parsing each raw value through `KvAssignment::from_str` to extract the key
56+
— and aggregates their counts.
57+
58+
For example, if `assistant.tool_choice=auto` has count 5 and
59+
`assistant.tool_choice=required` has count 2, the field
60+
`assistant.tool_choice` has an aggregate count of 7.
61+
62+
2. **Dedicated CLI arguments**: Arguments like `model` and `reasoning` map to
63+
known config field paths. The reverse mapping comes from the `CliRecord`
64+
infrastructure in [RFD 060] (each `CliRecord` has a `field` and `arg_id`
65+
pair). The wizard uses this to translate `model` usage into
66+
`assistant.model.id` ranking signal, and `reasoning` usage into
67+
`assistant.model.parameters.reasoning`.
68+
69+
Both sources are merged into a single ranking score per field path.
70+
71+
### Ranking heuristic
72+
73+
Fields are ranked by a combined score of frequency and recency:
74+
75+
```rust
76+
fn usage_score(count: u64, last_used: DateTime<Utc>, now: DateTime<Utc>) -> f64 {
77+
let days_ago = (now - last_used).num_days().max(0) as f64;
78+
let recency = 1.0 / (1.0 + days_ago / 7.0);
79+
let frequency = (count as f64).ln_1p();
80+
frequency * recency
81+
}
82+
```
83+
84+
This gives:
85+
86+
- A field used 30 times yesterday a higher score than one used 30 times a month
87+
ago.
88+
- A field used 5 times yesterday a higher score than one used once yesterday.
89+
- Logarithmic frequency scaling so that a field used 100 times isn't
90+
dramatically more prominent than one used 20 times.
91+
92+
The exact formula can be tuned based on real-world feedback. The important
93+
property is that both frequency and recency contribute, and neither dominates
94+
completely.
95+
96+
Fields with a score of zero (never used) are not included in the frecent tier —
97+
they remain in the natural-order tier.
98+
99+
### Signature change
100+
101+
The `interactive_config_browser` function gains a `CliUsage` parameter:
102+
103+
```rust
104+
fn interactive_config_browser(
105+
current: &PartialAppConfig,
106+
schema: &Schema,
107+
usage: &CliUsage,
108+
) -> Result<Vec<KvAssignment>>;
109+
```
110+
111+
The caller (in `run_inner()`) passes the `CliUsage` loaded from `Ctx`:
112+
113+
```rust
114+
if has_interactive_cfg(&cli.globals.config) {
115+
let schema = SchemaBuilder::build_root::<AppConfig>();
116+
let assignments = interactive_config_browser(&partial, &schema, &ctx.usage)?;
117+
// ...
118+
}
119+
```
120+
121+
### Field-path extraction
122+
123+
A helper function extracts per-field-path usage stats from `CliUsage`:
124+
125+
```rust
126+
fn field_usage_scores(
127+
usage: &CliUsage,
128+
command_path: &[&str],
129+
now: DateTime<Utc>,
130+
) -> HashMap<String, f64> {
131+
let Some(cmd) = usage.get_command(command_path) else {
132+
return HashMap::new();
133+
};
134+
135+
let mut scores: HashMap<String, f64> = HashMap::new();
136+
137+
// --cfg values: parse with KvAssignment to extract field paths.
138+
// The clap arg ID for `--cfg` is `config` (the Rust field name).
139+
// We use KvAssignment::from_str rather than splitting on '='
140+
// because the assignment syntax supports =, :=, +=, :+= etc.
141+
if let Some(cfg_arg) = cmd.args.get("config") {
142+
for (raw_value, stats) in &cfg_arg.values {
143+
if let Ok(kv) = KvAssignment::from_str(raw_value) {
144+
let score = usage_score(stats.count, stats.last_used, now);
145+
*scores.entry(kv.key_string()).or_default() += score;
146+
}
147+
}
148+
}
149+
150+
// Dedicated args: reverse-map arg ID to config field path.
151+
for (arg_id, arg_stats) in &cmd.args {
152+
if arg_id == "config" {
153+
continue; // already handled
154+
}
155+
156+
if let Some(field_path) = reverse_map_arg(command_path, arg_id) {
157+
let score = usage_score(arg_stats.count, arg_stats.last_used, now);
158+
*scores.entry(field_path.to_owned()).or_default() += score;
159+
}
160+
}
161+
162+
scores
163+
}
164+
```
165+
166+
The `reverse_map_arg` function uses the `CliRecord` registry from [RFD 060] to
167+
look up the config field path for a given clap argument ID and command.
168+
169+
### Visual indicators
170+
171+
The field selector uses distinct markers for each tier:
172+
173+
| Tier | Indicator | Example |
174+
|----------------------------|-----------|-----------------------------|
175+
| Configured (non-default) | `` | `● assistant.model.id` |
176+
| Wizard-edited this session | `` | `◆ style.reasoning.display` |
177+
| Frecent | `` | `◦ assistant.tool_choice` |
178+
| Remaining | (none) | ` assistant.name` |
179+
180+
## Drawbacks
181+
182+
- **Cold start**: A new workspace has no usage data. The wizard falls back to
183+
the two-tier ordering from [RFD 061] until enough invocations accumulate. This
184+
is by design — the wizard is useful without usage data, just not personalized.
185+
186+
- **Stale ranking**: If the user's workflow changes (e.g., switches models), the
187+
old model's flag still has high counts. The recency decay in the scoring
188+
formula mitigates this — unused fields fade over ~2-4 weeks — but don't
189+
disappear entirely. A future "reset usage" command could help.
190+
191+
## Alternatives
192+
193+
### Manual pinning instead of automatic ranking
194+
195+
Let users explicitly pin fields to the top of the wizard list. This gives full
196+
control but requires upfront configuration. Automatic ranking adapts without
197+
user effort.
198+
199+
These approaches aren't mutually exclusive — pinning could be layered on top of
200+
automatic ranking in a future RFD.
201+
202+
### Workspace-global ranking (not per-command)
203+
204+
Aggregate usage across all commands instead of per-command. Simpler, but less
205+
accurate: `--model` usage on `query` doesn't mean `assistant.model.id` should be
206+
prominent when configuring `conversation ls`.
207+
208+
## Non-Goals
209+
210+
- **Usage data recording**: This RFD consumes usage data; [RFD 062] handles
211+
recording it. The boundary is: RFD 062 writes, this RFD reads.
212+
213+
- **Usage data UI**: Displaying usage stats to the user (e.g., `jp usage show`)
214+
is out of scope. This RFD only uses usage data to improve wizard field
215+
ordering.
216+
217+
## Risks and Open Questions
218+
219+
- **Ranking tuning**: The scoring formula (logarithmic frequency × inverse
220+
recency) is a reasonable starting point but hasn't been validated with real
221+
usage data. The formula is easy to adjust without changing the architecture.
222+
223+
- **Reverse mapping completeness**: The `CliRecord` registry from [RFD 060] may
224+
not cover all arguments initially. Missing mappings mean those arguments don't
225+
contribute to field ranking — the wizard still works, just with less signal.
226+
This degrades gracefully.
227+
228+
- **Field renames**: Since [RFD 062] keys usage data by clap argument ID
229+
(derived from the Rust field name), renaming a field orphans its usage
230+
counters (see RFD 062's risk discussion). For this RFD, the consequence is
231+
that a renamed argument temporarily loses its ranking boost until enough new
232+
usage accumulates. This is a minor UX regression, not a failure.
233+
234+
## Implementation Plan
235+
236+
### Phase 1: Field-path extraction and scoring
237+
238+
1. Implement `field_usage_scores()` that reads `CliUsage` and produces a
239+
`HashMap<String, f64>` of field path → score.
240+
2. Implement `--cfg` value grouping by field path via `KvAssignment` parsing.
241+
3. Implement `reverse_map_arg()` using `CliRecord` from [RFD 060].
242+
4. Implement `usage_score()` with the frequency × recency formula.
243+
5. Unit tests with synthetic usage data.
244+
245+
Depends on: [RFD 062] Phase 1 (core types).
246+
247+
### Phase 2: Integration into wizard
248+
249+
1. Add `usage: &CliUsage` parameter to `interactive_config_browser()`.
250+
2. Insert frecent tier into field ordering logic.
251+
3. Add `` visual marker for the new tier.
252+
4. Pass `ctx.usage` from `run_inner()` into the wizard.
253+
254+
Depends on: Phase 1, [RFD 061] Phase 1 (core wizard loop).
255+
256+
Both phases can be merged as a single PR since the feature is small and
257+
self-contained.
258+
259+
## References
260+
261+
- [RFD 061]: Interactive config (the wizard this extends)
262+
- [RFD 060]: Config explain (`CliRecord` reverse mapping)
263+
- [RFD 062]: CLI usage tracking (provides the data)
264+
- `KvAssignment::from_str` in `jp_config/src/assignment.rs``KEY=VALUE`
265+
parsing used to extract field paths from `--cfg` values
266+
- `Globals.config` in `jp_cli/src/lib.rs` — the `--cfg` flag (clap arg ID:
267+
`config`)
268+
269+
[RFD 060]: 060-config-explain.md
270+
[RFD 062]: 062-cli-usage-tracking.md
271+
[RFD 061]: 061-interactive-config.md

0 commit comments

Comments
 (0)