|
| 1 | +# RFD 063: Usage-Based Wizard Field Ordering |
| 2 | + |
| 3 | +- **Status**: Discussion |
| 4 | +- **Category**: Design |
| 5 | +- **Authors**: Jean Mertz <git@jeanmertz.com> |
| 6 | +- **Date**: 2026-03-21 |
| 7 | + |
| 8 | +## Summary |
| 9 | + |
| 10 | +Extend the interactive config wizard ([RFD 061]) with a frecency-based tier in |
| 11 | +the field selector ordering, powered by the CLI usage tracking infrastructure |
| 12 | +from [RFD 062]. Fields the user frequently sets via `--cfg` or dedicated CLI |
| 13 | +flags are promoted in the list, making the wizard increasingly personalized over |
| 14 | +time. |
| 15 | + |
| 16 | +## Motivation |
| 17 | + |
| 18 | +[RFD 061] ships the interactive config wizard with a two-tier field ordering: |
| 19 | +configured (non-default) fields first, then all remaining fields in natural |
| 20 | +order. This is functional but static — a user who sets `assistant.tool_choice` |
| 21 | +on every other invocation sees it in the same position as a field they've never |
| 22 | +touched. |
| 23 | + |
| 24 | +With [RFD 062]'s usage tracking in place, the wizard has access to per-flag |
| 25 | +usage data: which `--cfg` fields and dedicated flags are used, how often, and |
| 26 | +when. This RFD adds a middle tier — "frecent" (frequently + recently used) — |
| 27 | +that promotes fields based on this data. |
| 28 | + |
| 29 | +## Design |
| 30 | + |
| 31 | +### Field ordering (updated) |
| 32 | + |
| 33 | +The wizard's field selector orders fields in three tiers: |
| 34 | + |
| 35 | +1. **Already configured** (non-default) fields — marked with a visual |
| 36 | + indicator (e.g., `●` prefix). |
| 37 | +2. **Frecent** fields — fields the user has set in previous |
| 38 | + invocations but that are currently at their default value. Marked with a |
| 39 | + distinct indicator (e.g., `◦` prefix or dimmed text). |
| 40 | +3. **All remaining** fields in their natural order (as returned by |
| 41 | + `AppConfig::fields()`). |
| 42 | + |
| 43 | +Fields configured during the current wizard session are also marked, distinct |
| 44 | +from fields configured by other layers. |
| 45 | + |
| 46 | +### Data sources |
| 47 | + |
| 48 | +The ranking signal comes from two places in the `CliUsage` data ([RFD 062]): |
| 49 | + |
| 50 | +1. **`--cfg` argument values**: The `values` map under |
| 51 | + `cli.commands.query.args.config` contains entries like |
| 52 | + `assistant.tool_choice=auto` (raw `KEY=VALUE` strings). The clap argument ID |
| 53 | + is `config` (the Rust field name in `Globals`), though users know it as |
| 54 | + `--cfg` / `-c`. The wizard groups these by field path on the read side — |
| 55 | + parsing each raw value through `KvAssignment::from_str` to extract the key |
| 56 | + — and aggregates their counts. |
| 57 | + |
| 58 | + For example, if `assistant.tool_choice=auto` has count 5 and |
| 59 | + `assistant.tool_choice=required` has count 2, the field |
| 60 | + `assistant.tool_choice` has an aggregate count of 7. |
| 61 | + |
| 62 | +2. **Dedicated CLI arguments**: Arguments like `model` and `reasoning` map to |
| 63 | + known config field paths. The reverse mapping comes from the `CliRecord` |
| 64 | + infrastructure in [RFD 060] (each `CliRecord` has a `field` and `arg_id` |
| 65 | + pair). The wizard uses this to translate `model` usage into |
| 66 | + `assistant.model.id` ranking signal, and `reasoning` usage into |
| 67 | + `assistant.model.parameters.reasoning`. |
| 68 | + |
| 69 | +Both sources are merged into a single ranking score per field path. |
| 70 | + |
| 71 | +### Ranking heuristic |
| 72 | + |
| 73 | +Fields are ranked by a combined score of frequency and recency: |
| 74 | + |
| 75 | +```rust |
| 76 | +fn usage_score(count: u64, last_used: DateTime<Utc>, now: DateTime<Utc>) -> f64 { |
| 77 | + let days_ago = (now - last_used).num_days().max(0) as f64; |
| 78 | + let recency = 1.0 / (1.0 + days_ago / 7.0); |
| 79 | + let frequency = (count as f64).ln_1p(); |
| 80 | + frequency * recency |
| 81 | +} |
| 82 | +``` |
| 83 | + |
| 84 | +This gives: |
| 85 | + |
| 86 | +- A field used 30 times yesterday a higher score than one used 30 times a month |
| 87 | + ago. |
| 88 | +- A field used 5 times yesterday a higher score than one used once yesterday. |
| 89 | +- Logarithmic frequency scaling so that a field used 100 times isn't |
| 90 | + dramatically more prominent than one used 20 times. |
| 91 | + |
| 92 | +The exact formula can be tuned based on real-world feedback. The important |
| 93 | +property is that both frequency and recency contribute, and neither dominates |
| 94 | +completely. |
| 95 | + |
| 96 | +Fields with a score of zero (never used) are not included in the frecent tier — |
| 97 | +they remain in the natural-order tier. |
| 98 | + |
| 99 | +### Signature change |
| 100 | + |
| 101 | +The `interactive_config_browser` function gains a `CliUsage` parameter: |
| 102 | + |
| 103 | +```rust |
| 104 | +fn interactive_config_browser( |
| 105 | + current: &PartialAppConfig, |
| 106 | + schema: &Schema, |
| 107 | + usage: &CliUsage, |
| 108 | +) -> Result<Vec<KvAssignment>>; |
| 109 | +``` |
| 110 | + |
| 111 | +The caller (in `run_inner()`) passes the `CliUsage` loaded from `Ctx`: |
| 112 | + |
| 113 | +```rust |
| 114 | +if has_interactive_cfg(&cli.globals.config) { |
| 115 | + let schema = SchemaBuilder::build_root::<AppConfig>(); |
| 116 | + let assignments = interactive_config_browser(&partial, &schema, &ctx.usage)?; |
| 117 | + // ... |
| 118 | +} |
| 119 | +``` |
| 120 | + |
| 121 | +### Field-path extraction |
| 122 | + |
| 123 | +A helper function extracts per-field-path usage stats from `CliUsage`: |
| 124 | + |
| 125 | +```rust |
| 126 | +fn field_usage_scores( |
| 127 | + usage: &CliUsage, |
| 128 | + command_path: &[&str], |
| 129 | + now: DateTime<Utc>, |
| 130 | +) -> HashMap<String, f64> { |
| 131 | + let Some(cmd) = usage.get_command(command_path) else { |
| 132 | + return HashMap::new(); |
| 133 | + }; |
| 134 | + |
| 135 | + let mut scores: HashMap<String, f64> = HashMap::new(); |
| 136 | + |
| 137 | + // --cfg values: parse with KvAssignment to extract field paths. |
| 138 | + // The clap arg ID for `--cfg` is `config` (the Rust field name). |
| 139 | + // We use KvAssignment::from_str rather than splitting on '=' |
| 140 | + // because the assignment syntax supports =, :=, +=, :+= etc. |
| 141 | + if let Some(cfg_arg) = cmd.args.get("config") { |
| 142 | + for (raw_value, stats) in &cfg_arg.values { |
| 143 | + if let Ok(kv) = KvAssignment::from_str(raw_value) { |
| 144 | + let score = usage_score(stats.count, stats.last_used, now); |
| 145 | + *scores.entry(kv.key_string()).or_default() += score; |
| 146 | + } |
| 147 | + } |
| 148 | + } |
| 149 | + |
| 150 | + // Dedicated args: reverse-map arg ID to config field path. |
| 151 | + for (arg_id, arg_stats) in &cmd.args { |
| 152 | + if arg_id == "config" { |
| 153 | + continue; // already handled |
| 154 | + } |
| 155 | + |
| 156 | + if let Some(field_path) = reverse_map_arg(command_path, arg_id) { |
| 157 | + let score = usage_score(arg_stats.count, arg_stats.last_used, now); |
| 158 | + *scores.entry(field_path.to_owned()).or_default() += score; |
| 159 | + } |
| 160 | + } |
| 161 | + |
| 162 | + scores |
| 163 | +} |
| 164 | +``` |
| 165 | + |
| 166 | +The `reverse_map_arg` function uses the `CliRecord` registry from [RFD 060] to |
| 167 | +look up the config field path for a given clap argument ID and command. |
| 168 | + |
| 169 | +### Visual indicators |
| 170 | + |
| 171 | +The field selector uses distinct markers for each tier: |
| 172 | + |
| 173 | +| Tier | Indicator | Example | |
| 174 | +|----------------------------|-----------|-----------------------------| |
| 175 | +| Configured (non-default) | `●` | `● assistant.model.id` | |
| 176 | +| Wizard-edited this session | `◆` | `◆ style.reasoning.display` | |
| 177 | +| Frecent | `◦` | `◦ assistant.tool_choice` | |
| 178 | +| Remaining | (none) | ` assistant.name` | |
| 179 | + |
| 180 | +## Drawbacks |
| 181 | + |
| 182 | +- **Cold start**: A new workspace has no usage data. The wizard falls back to |
| 183 | + the two-tier ordering from [RFD 061] until enough invocations accumulate. This |
| 184 | + is by design — the wizard is useful without usage data, just not personalized. |
| 185 | + |
| 186 | +- **Stale ranking**: If the user's workflow changes (e.g., switches models), the |
| 187 | + old model's flag still has high counts. The recency decay in the scoring |
| 188 | + formula mitigates this — unused fields fade over ~2-4 weeks — but don't |
| 189 | + disappear entirely. A future "reset usage" command could help. |
| 190 | + |
| 191 | +## Alternatives |
| 192 | + |
| 193 | +### Manual pinning instead of automatic ranking |
| 194 | + |
| 195 | +Let users explicitly pin fields to the top of the wizard list. This gives full |
| 196 | +control but requires upfront configuration. Automatic ranking adapts without |
| 197 | +user effort. |
| 198 | + |
| 199 | +These approaches aren't mutually exclusive — pinning could be layered on top of |
| 200 | +automatic ranking in a future RFD. |
| 201 | + |
| 202 | +### Workspace-global ranking (not per-command) |
| 203 | + |
| 204 | +Aggregate usage across all commands instead of per-command. Simpler, but less |
| 205 | +accurate: `--model` usage on `query` doesn't mean `assistant.model.id` should be |
| 206 | +prominent when configuring `conversation ls`. |
| 207 | + |
| 208 | +## Non-Goals |
| 209 | + |
| 210 | +- **Usage data recording**: This RFD consumes usage data; [RFD 062] handles |
| 211 | + recording it. The boundary is: RFD 062 writes, this RFD reads. |
| 212 | + |
| 213 | +- **Usage data UI**: Displaying usage stats to the user (e.g., `jp usage show`) |
| 214 | + is out of scope. This RFD only uses usage data to improve wizard field |
| 215 | + ordering. |
| 216 | + |
| 217 | +## Risks and Open Questions |
| 218 | + |
| 219 | +- **Ranking tuning**: The scoring formula (logarithmic frequency × inverse |
| 220 | + recency) is a reasonable starting point but hasn't been validated with real |
| 221 | + usage data. The formula is easy to adjust without changing the architecture. |
| 222 | + |
| 223 | +- **Reverse mapping completeness**: The `CliRecord` registry from [RFD 060] may |
| 224 | + not cover all arguments initially. Missing mappings mean those arguments don't |
| 225 | + contribute to field ranking — the wizard still works, just with less signal. |
| 226 | + This degrades gracefully. |
| 227 | + |
| 228 | +- **Field renames**: Since [RFD 062] keys usage data by clap argument ID |
| 229 | + (derived from the Rust field name), renaming a field orphans its usage |
| 230 | + counters (see RFD 062's risk discussion). For this RFD, the consequence is |
| 231 | + that a renamed argument temporarily loses its ranking boost until enough new |
| 232 | + usage accumulates. This is a minor UX regression, not a failure. |
| 233 | + |
| 234 | +## Implementation Plan |
| 235 | + |
| 236 | +### Phase 1: Field-path extraction and scoring |
| 237 | + |
| 238 | +1. Implement `field_usage_scores()` that reads `CliUsage` and produces a |
| 239 | + `HashMap<String, f64>` of field path → score. |
| 240 | +2. Implement `--cfg` value grouping by field path via `KvAssignment` parsing. |
| 241 | +3. Implement `reverse_map_arg()` using `CliRecord` from [RFD 060]. |
| 242 | +4. Implement `usage_score()` with the frequency × recency formula. |
| 243 | +5. Unit tests with synthetic usage data. |
| 244 | + |
| 245 | +Depends on: [RFD 062] Phase 1 (core types). |
| 246 | + |
| 247 | +### Phase 2: Integration into wizard |
| 248 | + |
| 249 | +1. Add `usage: &CliUsage` parameter to `interactive_config_browser()`. |
| 250 | +2. Insert frecent tier into field ordering logic. |
| 251 | +3. Add `◦` visual marker for the new tier. |
| 252 | +4. Pass `ctx.usage` from `run_inner()` into the wizard. |
| 253 | + |
| 254 | +Depends on: Phase 1, [RFD 061] Phase 1 (core wizard loop). |
| 255 | + |
| 256 | +Both phases can be merged as a single PR since the feature is small and |
| 257 | +self-contained. |
| 258 | + |
| 259 | +## References |
| 260 | + |
| 261 | +- [RFD 061]: Interactive config (the wizard this extends) |
| 262 | +- [RFD 060]: Config explain (`CliRecord` reverse mapping) |
| 263 | +- [RFD 062]: CLI usage tracking (provides the data) |
| 264 | +- `KvAssignment::from_str` in `jp_config/src/assignment.rs` — `KEY=VALUE` |
| 265 | + parsing used to extract field paths from `--cfg` values |
| 266 | +- `Globals.config` in `jp_cli/src/lib.rs` — the `--cfg` flag (clap arg ID: |
| 267 | + `config`) |
| 268 | + |
| 269 | +[RFD 060]: 060-config-explain.md |
| 270 | +[RFD 062]: 062-cli-usage-tracking.md |
| 271 | +[RFD 061]: 061-interactive-config.md |
0 commit comments