|
| 1 | +# Project Review and Delivery Tracker — 2026-07-18 |
| 2 | + |
| 3 | +This document records the July 2026 project review and tracks the work required to turn AI |
| 4 | +Coding Stack from a mostly static directory into a trusted, continuously maintained decision |
| 5 | +layer for AI coding tools and models. |
| 6 | + |
| 7 | +## North Star |
| 8 | + |
| 9 | +AI Coding Stack should optimize for trustworthy decisions rather than raw page or entry count: |
| 10 | + |
| 11 | +1. **Data layer:** sourced, dated, reviewable facts about tools, models, providers, and vendors. |
| 12 | +2. **Decision layer:** search, filters, comparisons, and use-case-oriented stack recommendations. |
| 13 | +3. **Distribution layer:** the website, versioned data exports/API, embeddable comparisons, and |
| 14 | + community contribution workflows. |
| 15 | + |
| 16 | +## Review Baseline |
| 17 | + |
| 18 | +Baseline captured on 2026-07-18: |
| 19 | + |
| 20 | +- 151 manifest files across six entity categories. |
| 21 | +- 12 supported locales with aligned translation file structures. |
| 22 | +- Last manifest change: 2026-01-26. |
| 23 | +- Newest model `releaseDate`: 2025-12-23. |
| 24 | +- Last successful production deployment and `main` CI run: 2026-01-28. |
| 25 | +- 14 open pull requests; 13 reported as blocked at review time. |
| 26 | +- 40 `as unknown as` type escapes under `src/` and `scripts/`. |
| 27 | +- 39 client-side source files; several product pages exceed 500 lines. |
| 28 | +- 33 models produce 528 pair combinations, or 6,336 localized comparison pages if every pair |
| 29 | + is statically generated. |
| 30 | +- Approximately 36–51% of non-English UI strings are byte-identical to English. This is a proxy |
| 31 | + for untranslated placeholders, not a definitive translation-quality score. |
| 32 | + |
| 33 | +## Status Legend |
| 34 | + |
| 35 | +- [ ] Planned |
| 36 | +- [~] In progress |
| 37 | +- [x] Completed |
| 38 | +- [!] Blocked or requires a product/operations decision |
| 39 | + |
| 40 | +## P0 — Restore Maintenance Throughput |
| 41 | + |
| 42 | +- [x] Remove the deleted `develop` branch from deployment workflows. |
| 43 | + - Acceptance: staging has an intentional manual or trunk-based trigger. |
| 44 | +- [x] Consolidate GitHub Stars automation into one workflow. |
| 45 | + - Acceptance: one schedule, correct `data/github-stars.json` path, generated output included, |
| 46 | + and changes result in a reviewable PR. |
| 47 | +- [x] Make scheduled URL failures observable and actionable. |
| 48 | + - Acceptance: a failed URL check creates or updates an issue and the workflow reports failure. |
| 49 | +- [x] Repair stale CODEOWNERS, issue templates, README, contributor guide, and script docs. |
| 50 | + - Acceptance: every referenced path and npm command exists; manifest examples validate. |
| 51 | +- [ ] Triage and merge/close the existing blocked dependency and data PR backlog. |
| 52 | + - Acceptance: `main` is current, required checks run on bot PRs, and no obsolete update PRs |
| 53 | + remain open. |
| 54 | +- [ ] Define a regular release and data-refresh cadence. |
| 55 | + - Acceptance: named owner, weekly data review, and monthly production release review. |
| 56 | + |
| 57 | +## P0 — Correctness and User Trust |
| 58 | + |
| 59 | +- [x] Fix provider links emitted by the search results page. |
| 60 | +- [x] Fix untranslated/raw labels and invalid inferred provider links in two-model comparisons. |
| 61 | +- [x] Replace hard-coded 2025 metadata and copyright values with current or timeless values. |
| 62 | +- [x] Complete and integrate `ModelCompareSelector` on model detail pages. |
| 63 | +- [x] Add missing sitemap coverage for important static, comparison, extension, and vendor pages. |
| 64 | +- [x] Remove randomized fallback data from `GitHubStarHistory`; failed requests now render nothing. |
| 65 | +- [x] Define the meaning and limitations of the Verified badge in public documentation. |
| 66 | + |
| 67 | +## P1 — Data Trust Layer |
| 68 | + |
| 69 | +- [x] Extend schemas with optional provenance and freshness fields. |
| 70 | + - Candidate fields: `sources`, `lastVerifiedAt`, `verifiedBy`, `confidence`, and field-level |
| 71 | + effective dates where pricing or benchmarks change independently. |
| 72 | +- [ ] Backfill provenance for all verified entries before treating verification as authoritative. |
| 73 | +- [ ] Generate a data-health report covering freshness, missing sources, broken URLs, relationship |
| 74 | + integrity, translation placeholders, and missing pricing/benchmark values. |
| 75 | +- [ ] Add freshness policies (for example 30/60/90-day review thresholds by field/category). |
| 76 | +- [ ] Automate change discovery from official sources, but keep human review before merge. |
| 77 | +- [ ] Populate a real changelog from manifest diffs instead of keeping an empty static file. |
| 78 | + |
| 79 | +## P1 — Product Experience |
| 80 | + |
| 81 | +- [x] Stop statically generating every possible two-model comparison pair. |
| 82 | +- [ ] Unify `/models/comparison` and `/models/compare/...` into one understandable comparison |
| 83 | + journey. |
| 84 | +- [ ] Add “compare” actions to cards and detail pages with persistent selected items. |
| 85 | +- [x] Search localized names/descriptions, capabilities, modalities, platforms, vendors, and types. |
| 86 | +- [ ] Add explicit use-case metadata and include it in search when the schema supports it. |
| 87 | +- [ ] Add guided selection filters: interface, budget, model freedom, privacy/local execution, |
| 88 | + platform, and team workflow. |
| 89 | +- [x] Make the ranking destination directly reachable from keyboard and mobile navigation. |
| 90 | +- [ ] Refocus the homepage on current data, recent changes, popular comparisons, and recommended |
| 91 | + stacks rather than primarily explaining the product. |
| 92 | + |
| 93 | +## P1 — Quality Gates |
| 94 | + |
| 95 | +- [x] Add route-contract tests for every manifest category. |
| 96 | +- [ ] Add browser-level smoke tests for search, navigation, detail pages, locale switching, and |
| 97 | + comparison selection. |
| 98 | +- [ ] Add automated accessibility checks for dialogs, menus, comparison tables, and mobile nav. |
| 99 | +- [x] Run generation in CI and fail when tracked generated output differs. |
| 100 | +- [x] Make the aggregate CI job fail on cancelled or skipped required jobs as well as failures. |
| 101 | +- [ ] Add a small security workflow/policy for dependency and application review. |
| 102 | + |
| 103 | +## P2 — Architecture and Scale |
| 104 | + |
| 105 | +- [ ] Make `ComparisonTable` generic and remove `Record<string, unknown>` call-site casting. |
| 106 | +- [ ] Extract shared comparison definitions for IDEs, CLIs, and extensions. |
| 107 | +- [ ] Use the manifest registry as the single source of truth for route bases in search, sitemap, |
| 108 | + navigation, and metadata generation. |
| 109 | +- [ ] Split large client components and move static derivations back to server components. |
| 110 | +- [x] Keep generated TypeScript artifacts tracked and enforce freshness in CI. |
| 111 | +- [ ] Replace duplicated metadata constants and hand-maintained examples with generated values. |
| 112 | + |
| 113 | +## P2 — Distribution and Growth |
| 114 | + |
| 115 | +- [ ] Publish versioned JSON exports or a small read-only API. |
| 116 | +- [ ] Offer embeddable comparison cards and freshness/verification badges. |
| 117 | +- [ ] Generate weekly ecosystem change summaries from reviewed manifest diffs. |
| 118 | +- [ ] Let contribution forms generate schema-valid pull requests instead of requiring contributors |
| 119 | + to hand-author large JSON documents. |
| 120 | +- [ ] Track decision-oriented product metrics: searches completed, comparisons started, outbound |
| 121 | + official-link clicks, stale entries resolved, and contribution lead time. |
| 122 | + |
| 123 | +## Baseline Findings |
| 124 | + |
| 125 | +These findings were captured before the implementation batch and are retained as historical review |
| 126 | +evidence; completed items are reflected in the checklists and implementation log above. |
| 127 | + |
| 128 | +- `src/app/[locale]/search/page.client.tsx` bypasses the manifest registry route mapping. |
| 129 | +- `src/app/[locale]/models/compare/[models]/page.client.tsx` renders several internal keys and |
| 130 | + English-only strings directly. |
| 131 | +- Model comparison provider URLs are inferred from display names, which does not work for values |
| 132 | + such as `Z.ai`, `KwaiKAT`, or non-provider vendors. |
| 133 | +- `src/app/[locale]/models/compare/[models]/page.tsx` generates model pairs with O(n²) growth. |
| 134 | +- `.github/workflows/deploy-staging.yml` listens to the removed `develop` branch. |
| 135 | +- Stars updates are duplicated across two workflows and one workflow references an obsolete path. |
| 136 | +- Scheduled URL validation combines `continue-on-error` with a generic `failure()` condition. |
| 137 | +- README and contributor documentation reference old framework versions, commands, paths, and |
| 138 | + manifest shapes. |
| 139 | +- Sitemap coverage excludes several valuable routes and two detail categories. |
| 140 | +- `METADATA_DEFAULTS.currentYear`, footer text, and model metadata contain hard-coded 2025 values. |
| 141 | +- `GitHubStarHistory` generates random chart data when its external API fails. |
| 142 | +- The test suite is strong on repository integrity but thin on user-visible behavior. |
| 143 | + |
| 144 | +## Implementation Log |
| 145 | + |
| 146 | +### 2026-07-18 |
| 147 | + |
| 148 | +- Captured the review baseline and agreed direction. |
| 149 | +- Switched active development back to `main`. |
| 150 | +- Restored staging, scheduled URL checks, Stars refresh ownership, CI aggregation, and generated |
| 151 | + source drift checks. |
| 152 | +- Corrected contributor-facing paths, commands, runtime versions, script docs, and issue templates. |
| 153 | +- Fixed search routing and coverage, comparison localization/provider routing/pricing, sitemap |
| 154 | + coverage, current-year handling, and randomized chart fallback data. |
| 155 | +- Integrated the model comparison selector and moved pair pages to on-demand ISR instead of |
| 156 | + prebuilding every localized pair. |
| 157 | +- Added structured provenance/freshness fields and documented the Verified badge rollout without |
| 158 | + inventing source history for legacy records. |
| 159 | +- Preserved cached GitHub star values on transient API failures instead of overwriting them with |
| 160 | + `null`. |
| 161 | +- Added route/search regression tests and completed production verification: Biome, TypeScript, |
| 162 | + 25 tests, manifest validation, generation, Next.js build, and OpenNext Cloudflare bundling. |
0 commit comments