Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,18 @@ Directory scan / shared path filter (built-in skip lists + `.gitignore` / `.cdid
→ Populate FTS5 index
```

Explicit numeric `index` options are validated before database setup or worker
startup. The inclusive ranges are `--parallelism` 1..16,
`--max-file-bytes` 1..2147483647 bytes, `--max-symbols-per-file` 1..50000,
`--max-references-per-file` 1..1000000, `--dry-run-path-limit` 1..1000000,
`--watch-pending-path-limit` 1..262144, and `--debounce` 0..60000 ms. Any
explicit zero outside those ranges, negative, overflowed, non-numeric, or
over-limit value returns structured `E010_USAGE_ERROR` output and exit code 1;
an invalid occurrence is not hidden by a later duplicate. For compatibility,
invalid `CDIDX_INDEX_PARALLELISM`, `CDIDX_MAX_FILE_BYTES`, and
`CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` values still warn and fall back or clamp,
but JSON warnings identify the environment source and the effective value.

Every loaded text file is normalized and analyzed in one UTF-16 walk. The
resulting `NormalizedContentFacts` is the shared source for normalized line
count, oversized-line and FTS-token diagnostics, conflict-marker detection,
Expand Down Expand Up @@ -4220,6 +4232,17 @@ query コマンドも JSON profile block 用の `--profile` と command-scoped p
→ FTS5インデックス反映
```

明示された数値 `index` オプションは、database setup や worker 起動より前に検証します。
許容範囲(両端を含む)は `--parallelism` が 1..16、`--max-file-bytes` が
1..2147483647 byte、`--max-symbols-per-file` が 1..50000、
`--max-references-per-file` が 1..1000000、`--dry-run-path-limit` が
1..1000000、`--watch-pending-path-limit` が 1..262144、`--debounce` が
0..60000 ms です。範囲外のゼロ、負数、overflow、非数値、上限超過を明示すると、
構造化された `E010_USAGE_ERROR` と exit code 1 を返し、後続の重複指定が有効でも先行する
不正値を隠しません。互換性のため、不正な `CDIDX_INDEX_PARALLELISM`、
`CDIDX_MAX_FILE_BYTES`、`CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` は引き続き警告して
fallback または clamp しますが、JSON warning には環境変数由来であることと実効値を含めます。

読み込んだ text file は UTF-16 上の1回の走査で正規化と解析を行います。得られた
`NormalizedContentFacts` を、正規化後の行数、長すぎる行 / FTS token の診断、conflict
marker、replacement character の件数 / 行、80行・10行 overlap の chunk slice に対する
Expand Down
4 changes: 4 additions & 0 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -780,6 +780,8 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result
- `.github/scripts/configure-windows-test-host.ps1`
The `dotnet.yml` and `release.yml` Windows lanes share temp pinning and Defender exclusion setup here so both workflows keep the same test-host performance assumptions. General `TMP` / `TEMP` point to the runner's fast `RUNNER_TEMP\cdidx-temp` storage. Executable plugin, hook, and Git fixtures instead use `USERPROFILE\cdidx-trusted-test-temp`, whose protected current-user ACL and trusted ancestor chain satisfy the production executable-boundary contract; the script publishes this separate root as `CDIDX_TEST_TRUSTED_TEMP_ROOT`. Do not move ordinary SQLite or filesystem fixtures into that protected root, because placing the entire suite on the system drive materially increases Windows runtime. The script includes both roots in its normalized, de-duplicated Defender audit. When the `WinDefend` service is running, it submits the resulting string array in one `Add-MpPreference` invocation, reads Defender preferences back, and fails if any path is missing. When the service is stopped or unavailable, exclusions are irrelevant and the script emits a warning while continuing host setup; other Defender configuration or verification failures remain fatal. Update `CiWorkflowTests` when changing this split, batching, availability, audit, verification, or workflow call contract.
- The `dotnet.yml` SDK setup has one conditional retry for transient SDK download failures. Keep the first attempt marked `continue-on-error` only while the retry is guarded by its failed outcome, so a second failure still fails the job.
- `IndexCommandRunnerTests.cs`
Numeric `index` option coverage owns inclusive boundaries, zero/negative/overflow/non-numeric forms, duplicate precedence, text and JSON `E010_USAGE_ERROR` output before database mutation, and structured environment fallback warnings with source/effective-value provenance. Keep every numeric option in the shared matrix when its range or parsing changes.
- `DbRecoveryTests.cs`
Database corruption recovery and graceful degradation behavior. Filesystem setup failures for `cdidx index` (read-only DB files and unwritable DB parent directories) are covered in `IndexCommandRunnerTests.cs` so they exercise the same CLI JSON/stderr boundary users see.
- `Issue4857ManagedRestoreBackupTests.cs`
Expand Down Expand Up @@ -1867,6 +1869,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
primary の `dotnet publish` 出力と `CodeIndex` build artifact は main branch への push または手動 dispatch でだけ materialize し、pull request では作成しません。coverage upload は `TestResults/**/coverage.cobertura.xml` が存在するときだけ起動し、そのfileをfailure向け`TestResults` artifactから除外して、TRX・text log・その他のXML blame evidenceを残しながらcoverageを1回だけ保存します。
- `.github/scripts/configure-windows-test-host.ps1`
`dotnet.yml` と `release.yml` の Windows lane は、temp 固定と Defender 除外 setup をこのスクリプトで共有します。通常の `TMP` / `TEMP` は runner の高速な `RUNNER_TEMP\cdidx-temp` を使います。実行可能な plugin / hook / Git fixture だけは `USERPROFILE\cdidx-trusted-test-temp` を使い、current-user 限定の protected ACL と trusted な祖先 chain で production の executable-boundary contract を満たします。この専用 root は `CDIDX_TEST_TRUSTED_TEMP_ROOT` として helper へ渡します。Windows の実行時間を大きく増やすため、通常の SQLite / filesystem fixture を protected root へ移してはいけません。スクリプトは両 root を含む候補 path を正規化・重複排除して監査表示します。`WinDefend` service が稼働中なら、生成した string array を1回の `Add-MpPreference` 呼び出しで登録し、Defender preference を読み戻して欠けた path があれば失敗します。service が停止中または利用不能なら除外は意味を持たないため、warning を出して host setup を継続します。それ以外の Defender 設定失敗や検証失敗は引き続き fatal です。この split、batching、availability、audit、verification、または workflow 呼び出し contract を変更するときは `CiWorkflowTests` も更新してください。
- `IndexCommandRunnerTests.cs`
数値 `index` オプションの coverage は、両端を含む境界値、ゼロ / 負数 / overflow / 非数値、重複指定の優先順位、database mutation 前の text / JSON `E010_USAGE_ERROR`、および source / 実効値の provenance を持つ構造化環境変数 fallback warning を扱います。範囲や parse を変更するときは、共有 matrix にすべての数値オプションを残してください。
- `DbRecoveryTests.cs`
DB破損からの復旧とグレースフル劣化のテスト。`cdidx index` の filesystem setup failure(read-only DB file や書き込み不可の DB 親ディレクトリ)は、ユーザーが見る CLI JSON/stderr 境界を通すため `IndexCommandRunnerTests.cs` で扱います。
- `Issue4857ManagedRestoreBackupTests.cs`
Expand Down
25 changes: 25 additions & 0 deletions changelog.d/unreleased/5097.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
category: fixed
issues:
- 5097
affected:
- src/CodeIndex/Cli/IndexCommandRunner.Parse.cs
- src/CodeIndex/Cli/IndexCommandRunner.cs
- src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs
- src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs
- src/CodeIndex/Cli/IndexCommandRunner.Update.cs
- src/CodeIndex/Cli/IndexWatchJsonContracts.cs
- src/CodeIndex/Cli/IndexWatchRunner.Reporting.cs
- src/CodeIndex/Cli/CliFlagSchema.cs
- tests/CodeIndex.Tests/IndexCommandRunnerTests.cs
- DEVELOPER_GUIDE.md
- TESTING_GUIDE.md
---

## English

- **Invalid explicit numeric `index` options now fail before database setup (#5097)** — zero outside an option's inclusive range, negative, overflowed, non-numeric, and over-limit CLI values return structured `E010_USAGE_ERROR` output instead of warning and silently using a fallback. Environment-variable compatibility fallbacks remain warnings and now report their source and effective value in JSON output.

## 日本語

- **明示された不正な数値 `index` オプションを database setup 前に拒否するようになりました (#5097)** — オプションの許容範囲外のゼロ、負数、overflow、非数値、上限超過の CLI 値は、警告後に暗黙の fallback を使わず、構造化された `E010_USAGE_ERROR` を返します。環境変数の互換 fallback は警告として維持し、JSON 出力に source と実効値を含めます。
14 changes: 7 additions & 7 deletions src/CodeIndex/Cli/CliFlagSchema.cs
Original file line number Diff line number Diff line change
Expand Up @@ -649,7 +649,7 @@ private static IReadOnlyList<CliFlag> BuildAll()
Safety = CliOptionSafety.Preview,
},
new() { Name = "--show-paths", Description = "Show resolved local paths in maintenance diagnostics, recovery commands, or status --config output", PrimaryCommands = Set(RecoveryPathCommands.Concat(["index", "backfill-fold", "optimize", "vacuum", "db", "status"]).ToArray()) },
new() { Name = "--dry-run-path-limit", ValuePlaceholder = "<n>", Description = "Dry run only: candidate path processing limit before truncated lower-bound estimates", PrimaryCommands = Set("index") },
new() { Name = "--dry-run-path-limit", ValuePlaceholder = "<n>", Description = "Dry run only: candidate path processing limit (1..1000000); explicit invalid values fail before database setup", PrimaryCommands = Set("index") },
new() { Name = "--checkpoint", Description = "Create a DB checkpoint even when backfill preflight is already complete", PrimaryCommands = Set("backfill-fold") },
new() { Name = "--no-checkpoint", Description = "Skip the automatic DB checkpoint before a required backfill mutation", PrimaryCommands = Set("backfill-fold") },
new()
Expand All @@ -668,17 +668,17 @@ private static IReadOnlyList<CliFlag> BuildAll()
Safety = CliOptionSafety.Override,
},
new() { Name = "--duration-format", ValueDomain = Values(["auto", "seconds", "hms"]), Description = "Index elapsed time display format", PrimaryCommands = Set("index") },
new() { Name = "--max-file-bytes", ValuePlaceholder = "<bytes>", Description = "Override the per-file indexing size limit", PrimaryCommands = Set("index") },
new() { Name = "--max-symbols-per-file", ValuePlaceholder = "<n>", Description = "Skip file content, symbols, and references when one file emits too many symbols (max 50000)", PrimaryCommands = Set("index") },
new() { Name = "--max-references-per-file", ValuePlaceholder = "<n>", Description = "Skip references when one file emits too many references (max 1000000)", PrimaryCommands = Set("index") },
new() { Name = "--parallelism", ValuePlaceholder = "<n>", Description = "Full-scan and authoritative C# update extraction worker count (default: CPU count capped at 8; explicit max: 16; also honors CDIDX_INDEX_PARALLELISM)", PrimaryCommands = Set("index") },
new() { Name = "--max-file-bytes", ValuePlaceholder = "<bytes>", Description = "Per-file indexing size limit (1..2147483647 bytes; optional B/K/M/G suffix); invalid CDIDX_MAX_FILE_BYTES warns and uses the built-in default", PrimaryCommands = Set("index") },
new() { Name = "--max-symbols-per-file", ValuePlaceholder = "<n>", Description = "Skip file content, symbols, and references when one file emits too many symbols (1..50000)", PrimaryCommands = Set("index") },
new() { Name = "--max-references-per-file", ValuePlaceholder = "<n>", Description = "Skip references when one file emits too many references (1..1000000)", PrimaryCommands = Set("index") },
new() { Name = "--parallelism", ValuePlaceholder = "<n>", Description = "Extraction worker count (1..16; default CPU count capped at 8); invalid CDIDX_INDEX_PARALLELISM warns and uses the documented effective value", PrimaryCommands = Set("index") },
new() { Name = "--memory-trace", Description = "Include phase memory samples in index JSON output", PrimaryCommands = Set("index") },
new() { Name = "--commits", ValuePlaceholder = "<commit-ref>", Description = "Update files changed in given git commits", PrimaryCommands = Set("index") },
new() { Name = "--changed-between", ValuePlaceholder = "<old-ref> <new-ref>", Description = "Update files changed between two git refs", PrimaryCommands = Set("index") },
new() { Name = "--files", ValuePlaceholder = "<path>", ValueKind = CliOptionValueKind.FilePath, Description = "Update explicit files after validating the full selection atomically; invalid or duplicate paths reject before writes, while indexed missing paths remain deletion targets", PrimaryCommands = Set("index") },
new() { Name = "--watch", Description = "Continuous reindex on file changes (rejects --commits / --changed-between / --files / --dry-run)", PrimaryCommands = Set("index") },
new() { Name = "--debounce", ValuePlaceholder = "<ms>", Description = "Watch only: coalesce file events into one update after <ms> of quiet (default 500)", PrimaryCommands = Set("index") },
new() { Name = "--watch-pending-path-limit", ValuePlaceholder = "<n>", Description = "Watch only: changed-path queue limit before full-rescan fallback", PrimaryCommands = Set("index") },
new() { Name = "--debounce", ValuePlaceholder = "<ms>", Description = "Watch only: coalesce file events after 0..60000 ms of quiet (default 500)", PrimaryCommands = Set("index") },
new() { Name = "--watch-pending-path-limit", ValuePlaceholder = "<n>", Description = "Watch only: changed-path queue limit (1..262144); invalid CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT warns and uses the documented effective value", PrimaryCommands = Set("index") },
new() { Name = "--output", ShortName = "-o", ValuePlaceholder = "<path>", ValueKind = CliOptionValueKind.FilePath, Description = "Report bundle or suggestions export output path", PrimaryCommands = Set("report", "suggestions") },
new() { Name = "--redact-paths", Description = "Redact machine-specific paths (the default for recovery/config JSON and reports)", PrimaryCommands = Set(RecoveryPathCommands.Concat(["status", "report"]).ToArray()) },
new() { Name = "--no-log", Description = "Exclude global tool log from bundle", PrimaryCommands = Set("report") },
Expand Down
6 changes: 4 additions & 2 deletions src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,10 @@ private static int RunDryRun(
bool authoritativeFullScan;
var errorSamples = new List<CliJsonMessage>();
var errorCount = 0;
var warningSamples = new List<CliJsonMessage>();
var warningCount = 0;
var warningSamples = options.OptionWarnings
.Take(DryRunWarningSampleLimit)
.ToList();
var warningCount = options.OptionWarnings.Count;
var dryScanErrorKeys = new HashSet<string>(StringComparer.Ordinal);
DryRunScanMetadata dryScanMetadata;
DryRunDbSnapshot dbSnapshot;
Expand Down
1 change: 1 addition & 0 deletions src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ void ThrowIfFullScanCancelled(int filesProcessed, int? filesTotal)
})
.ToList();
var warningList = discovery.WarningList;
warningList.InsertRange(0, options.OptionWarnings);
AddProjectMarkerFingerprintWarnings(currentHotspotFamilyMarkerFingerprints, warningList, options);
var scanCheckpointPath = discovery.ScanCheckpointPath;
if (options.MemoryTrace)
Expand Down
Loading
Loading