Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

[Seungpyo1007]

#1

-build the curated TechAPI dataset across every category (brand, cpu, gpu, smartphone, soc) and ship the supporting validator / model / docs / site updates that go with it. New layout, broader manufacturer coverage (1989 → 2026), and a passing python -m scripts.validate for all categories.

This issue is the single rollout tracker; each phase below maps to one or more commits.

Plan

Phase 1 — Schema + validator updates

• Widen scripts/validate.py cores 1-256 → 1-512 and threads 1-512 → 1-1024 (Clearwater Forest 288C)
• Widen scripts/validate.py process_nm upper bound so pre-14 nm SoCs (Apple A7 28 nm, Samsung Exynos 4210 45 nm, etc.) validate
• Update scripts/seed.py for the new layout
• Update app/models/gpu.py and app/schemas/gpu.py for any new GPU fields

Phase 2 — Directory restructure

• Split data/cpu/<mfr>/<year>/*.json into data/cpu/<mfr>/<year>/<consumer|enterprise>/*.json (consumer = desktop+laptop, enterprise = hedt+server)

Phase 3 — Brand catalog expansion

• Add brand records for IBM, Motorola, DEC, Cyrix, Transmeta, VIA, Ampere, Hygon, Loongson, Zhaoxin, Microsoft, ARM, 3dfx, Matrox, SiS, plus mobile/laptop OEMs (Acer, ASUS, Lenovo, MSI, Razer, Sony, Sharp, Panasonic, LG, etc.)

Phase 4 — CPU dataset (1989-2026, 464 records)

• AMD Ryzen 1000 desktop + Threadripper 1000 (2017)
• AMD Ryzen 2000/3000 desktop + APU + Threadripper 2000/3000 (2018-2020)
• AMD Ryzen 4000-9000 desktop / X3D / AI Max / 9000F (2020-2026)
• AMD Ryzen U-series + H/HS/HX mobile (Raven Ridge → Strix Halo PRO)
• AMD Threadripper 5000/7000/9000 + PRO WX-series
• AMD EPYC Naples → Turin (incl. Bergamo 9754, Turin 9965/9755/9985)
• AMD Athlon Zen / A-series Kaveri & Bristol Ridge / FX-4/6/8/9 / Phenom II / Athlon II / Opteron
• Intel 80486 → Pentium / Pentium Pro / Pentium II/III/M / Pentium 4
• Intel Core 2 / Core Duo / Core Solo / Core 2 Extreme QX
• Intel Core i 1st-14th gen desktop incl. KF/F + mobile QM/HQ/U/H/HK
• Intel Core Ultra 100/200/300 (Meteor / Arrow / Lunar / Panther Lake)
• Intel Xeon E3/E5 historical
• Intel Xeon Scalable Bronze/Silver/Gold/Platinum + Xeon W + Xeon CPU Max (HBM SPR)
• Intel Xeon 6 Granite Rapids + Sierra Forest + Clearwater Forest 288C
• Intel Atom + N-series + Pentium G / Celeron G
• IBM POWER1 → POWER11 server lineup (1990-2025)
• IBM z mainframe z10 → Telum II z17
• IBM PowerPC G5 970/970FX/970MP, G3 750/750CXe, Cell BE, PowerXCell 8i
• IBM console CPUs: Gekko, Xenon, Broadway, Espresso (GameCube/Xbox 360/Wii/Wii U)
• Motorola/Freescale PowerPC 601/603/604/604e + 7400/7450/7455/7457/7447A/7448 (G4)
• DEC Alpha 21064/21164/21264
• Cyrix 6x86/MII, Transmeta Crusoe/Efficeon, VIA C3/Nano, Zhaoxin KX
• Ampere Altra Max + AmpereOne, NVIDIA Grace, Microsoft Cobalt 100, Qualcomm Snapdragon X/X2, Hygon, Loongson 3A6000

Phase 5 — GPU dataset

• NVIDIA GeForce / Quadro / RTX consumer + datacenter (Pascal → Blackwell)
• AMD Radeon RX 5000/6000/7000/9000 consumer + Instinct datacenter
• Intel Arc Alchemist + Battlemage
• Apple GPU references (deferred to SoC scope)
• Legacy: 3dfx Voodoo, Matrox, SiS

Phase 6 — Smartphone dataset

• Samsung Galaxy S / Note / Z series
• Apple iPhone (model entries)
• Xiaomi / Redmi / Poco
• Google Pixel
• OnePlus / Oppo / Vivo / iQOO / Realme / Nothing / Honor / Motorola / Sony / Asus ROG Phone / Razer Phone

Phase 7 — SoC dataset

• Qualcomm Snapdragon 8 Gen 1/2/3, 8 Elite + mid-range 7-series
• MediaTek Dimensity 8000/9000 series
• Apple A-series (A7 → A18 Pro) with widened process_nm range
• Samsung Exynos 2200/2400 + classic Exynos 4210/4412
• Google Tensor G1-G5
• HiSilicon Kirin, Unisoc, Allwinner, Amlogic, Rockchip

Phase 8 — Docs + site

• Update README.md data summary
• Update data/README.md with new consumer/enterprise convention
• Update site/src/pages/index.astro to reflect dataset size

Phase 9 — Validation gate

• python -m scripts.validate exits 0 for all categories

Result target

• 15+ CPU manufacturers, 464+ CPU records
• 30+ GPU brands, full Pascal → Blackwell + RDNA → RDNA 5 coverage
• Comprehensive smartphone + SoC catalog
• All validators green

[Seungpyo1007]

Rolled out in 11 commits on main

#	Commit	Subject
1	4dc4b57	chore(scripts): widen validator bounds and refresh seed loader
2	9708ee3	feat(app): expand GPU model and schema for richer hardware specs
3	7a9df9e	feat(data/brand): restructure into country folders and expand catalog
4	de9b32c	feat(data/cpu): expand AMD lineup from K5 to Zen 6 with segment split
5	b08e2cf	feat(data/cpu): expand Intel lineup from i486 to Panther Lake with segment split
6	9781670	feat(data/cpu): add IBM POWER, z mainframe, PowerPC, Cell, Motorola G3/G4
7	5cd4b5b	feat(data/cpu): add historic and niche vendors across 11 manufacturers
8	2a434bf	feat(data/gpu): rebuild discrete GPU catalog across 10 vendors with segment split
9	fc7bcd1	feat(data/smartphone): rebuild smartphone catalog across major brands
10	54306c8	feat(data/soc): rebuild SoC catalog covering Apple, Qualcomm, MediaTek, Samsung, Google, HiSilicon
11	f64a74a	docs: update README and data/README for expanded dataset and new layout

(The site homepage update was already on main as ef15e00 feat(site): add interactive API playground and redesign homepage, so the rebase deduplicated it.)

Result

• ✅ python -m scripts.validate PASS across brand / cpu / gpu / smartphone / soc
• 464 CPU records, 330 GPU records, ~366 smartphone records, ~100+ SoC records, 129 brand records
• New layouts: brand/<country>/<slug>.json + cpu/<mfr>/<year>/<consumer|enterprise>/<slug>.json
• Tracked in Project: TechAPI Dataset Rebuild