Skip to content

PLAN.md

Latest commit

 

History

History
210 lines (183 loc) · 12.8 KB

PLAN.md

File metadata and controls

210 lines (183 loc) · 12.8 KB
Raw

Yuri's Revenge (gamemd.exe) AI-Driven Decompilation Plan

Goal: a matching decompilation of gamemd.exe (C&C Red Alert 2: Yuri's Revenge 1.001), driven by an autonomous AI agent loop, accelerated by the Ares-Developers/YRpp reverse-engineering headers.

Target binary: C:\Games\Steam\steamapps\common\Command & Conquer Red Alert II\gamemd.exe (5,286,208 bytes — the 1.001 binary that YRpp/Ares/Phobos all target; the installed Phobos.dll confirms the address map applies to this exact executable).

1. Toolchain choice: why reccmp, not objdiff

Some AI-driven decomp projects for GameCube-era games use PowerPC/Metrowerks CodeWarrior toolchains, split the binary into objects with decomp-toolkit, and score per-object matches with objdiff. None of that applies to a Win32 PE. The x86/MSVC decomp ecosystem has its own standard: reccmp (from the LEGO Island decomp), which annotates source functions with original addresses, recompiles the whole EXE with a period-correct MSVC, and diffs the recompiled binary + PDB against the original, producing per-function match percentages and HTML reports.

Concern Choice for this project
Arch / compiler x86 / MSVC 6.0 (verify via PE Rich header)
Match scoring reccmp, per-function vs original EXE
Symbol source YRpp address map (huge head start)
Build configure.py + ninja driving VC6 CL.EXE
Agent loop headless claude -p in a scripted retry loop

Overall structure, borrowed from the broader AI-driven-decomp pattern that's proven out on other titles:

  • Repo layout (src/, include/, orig/, config/, tools/, docs/, .claude/).
  • A CLAUDE.md runbook: "pick a target with leverage, recover the dependency cluster, verifier is the final judge, commit only on material improvement, never hardcode addresses or invent junk names."
  • agentic_loop.py: infinite loop, spawn agent with the runbook prompt, hard timeout (~25 min), 10 s pause, repeat.

2. Why YRpp is the force multiplier

YRpp is not a decompilation — it's a hooking SDK. Its headers declare game classes with exact field offsets, vtable layouts, and thousands of member functions bound to absolute addresses via JMP_THIS(0x......)-style macros. There are no function bodies to copy, but it hands us, pre-verified by 20 years of community modding:

  1. A symbol map: scrape every JMP_THIS/JMP_STD/DEFINE_* macro → address → mangled class::method(signature) table covering a large fraction of the game-logic code. This replaces what would otherwise be an entire manual symbol-recovery phase, done for free.
  2. Struct/class layouts: field names, types, and offsets for AbstractClass, TechnoClass, UnitClass, HouseClass, RulesClass, etc., including inheritance chains and virtual table ordering — exactly what the AI agent needs to turn Ghidra pseudo-C into plausible original C++.
  3. Enums and constants (armor types, mission enums, INI keys) that make decompiled switch statements readable.
  4. Static/global object addresses (RulesClass::Instance etc.) for data labeling.

Additional accelerants from the same ecosystem:

  • Ares and Phobos source: both reimplement or heavily document specific game routines in their hooks — useful reading to understand what a function is supposed to do, not a source to copy from (see license note below).
  • Community IDA databases for gamemd.exe circulate in the modding community (CnCNet / ModEnc circles) — worth acquiring; they carry additional names YRpp lacks.

Decided against: EA's 2025 GPL source releases (Tiberian Dawn, Red Alert 1, Renegade, Generals) as a seed source for shared Westwood library code. Not worth the licensing entanglement for this project — all game-logic source is written independently by reading the decompiled output, never transcribed from GPL reference material.

License: this project is released under CC0 1.0 Universal (see LICENSE) — applies to the reconstructed source we write ourselves. YRpp is used strictly as a factual reference (addresses, struct offsets, function signatures extracted into config/symbols.csv/config/globals.csv) — never by copying its header text verbatim into our own headers, which keeps YRpp's own (unclear) licensing out of the equation. Same rule for Ares/Phobos: read for semantic understanding, reimplement independently.

3. Repository layout

YRdecomp/
├── PLAN.md                  # this file
├── CLAUDE.md                # agent runbook (the AGENTS.md equivalent)
├── configure.py             # generates build.ninja (VC6 CL flags per TU)
├── reccmp-project.yml       # original binary path, recompiled EXE+PDB paths
├── agentic_loop.py          # autonomous loop: spawn agent, timeout, repeat
├── orig/                    # gamemd.exe copy + SHA256 (gitignored)
├── include/
│   ├── yrpp/                # YRpp as git submodule (reference/types)
│   └── game/                # our reconstructed headers (what we ship)
├── src/                     # reconstructed .cpp, one per original TU/class
├── stubs/                   # auto-generated stubs so the EXE always links
├── ghidra/                  # headless project + export scripts (gitignored DB)
├── tools/
│   ├── yrpp_symbols.py      # scrape YRpp macros → symbols.csv (addr,name,sig)
│   ├── export_function.py   # Ghidra headless: pseudo-C + asm for one address
│   ├── gen_stubs.py         # emit stubs for every not-yet-decompiled function
│   ├── pick_target.py       # choose next function/cluster for the agent
│   └── report.py            # aggregate reccmp output → progress dashboard
├── config/                  # VC6 flag sets, section maps, TU→address ranges
└── docs/                    # findings: compiler flags, calling conventions, CRT notes

4. Phases

Phase 0 — Ground truth and toolchain (~days)

  1. Copy gamemd.exe into orig/, record SHA256; confirm it is the unprotected 1.001 binary (loads cleanly in Ghidra, .text not packed).
  2. Read the PE Rich header to pin the exact compiler/linker builds (expected: VC6, possibly with a service pack). Record in docs/compiler.md.
  3. Install the period compiler: portable MSVC 6.0 (CL 12.00) — runs fine on Win11. Get cvdump.exe (microsoft-pdb repo) for reccmp's PDB parsing.
  4. pip install reccmp; install Ghidra + ninja; init git repo, add YRpp submodule.
  5. Smoke test: compile a trivial .cpp with VC6, link, run reccmp against orig/gamemd.exe with one hand-annotated function, confirm the report pipeline works end to end.

Phase 1 — Symbol map and analysis DB (~days)

  1. tools/yrpp_symbols.py: parse all YRpp headers, emit config/symbols.csv (address, C++ name, signature, class, calling convention). Also extract class field layouts into a machine-readable config/types.json.
  2. Ghidra headless: import gamemd.exe, run auto-analysis once, then a script that applies symbols.csv names and (via Ghidra's C parser) YRpp types. Save the project — this is the agent's oracle.
  3. Enumerate all functions Ghidra found; join against symbols.csv. Output config/functions.csv with columns: address, size, name (or sub_XXXXXX), YRpp-known?, callers/callees, status (unstarted/stub/partial/matching).
  4. Partition the address space into planned translation units (config/tu_map.yml): YRpp's per-class headers give the natural boundaries (one TU per game class, plus CRT, WWLib, DirectDraw wrapper, etc. buckets).

Phase 2 — Always-linkable skeleton — DONE

  1. tools/gen_stubs.py: generated one stub per function (all 8,584) across 135 TU .cpp files under src/, each tagged // FUNCTION: GAMEMD 0x....... Stub bodies are free functions, not full class reconstructions — reccmp only needs address + PDB debug-line correlation to match, not real signatures; those get reconstructed per-function as each one is actually decompiled.
  2. configure.py emits build.ninja: all 135 TUs compiled with VC6 (/O2 /Ob1 /Z7 — validated exact-match flags, see docs/compiler.md), linked into build/gamemd_recomp.dll with a PDB. Gotchas from bringing this up the first time (ninja's Windows command execution, LIB env var vs /LIBPATH:, /Zi vs /Z7 for parallel builds, inline functions vanishing from the PDB) are in docs/build_notes.md.
  3. Baseline reccmp run, via reccmp-project.yml/reccmp-user.yml/reccmp-build.yml: Implemented 100.00% (8584/8584), Accuracy 0.37% — exactly the expected starting point (everything present and annotated, nothing real written yet). This is the number Phase 3 climbs function by function.
  4. Compiler flags validated empirically ahead of schedule, during the Phase 0.5 smoke test (tools/smoketest/GetProgress.cpp) — 100% byte-identical match achieved before the full skeleton was even generated.

Phase 3 — The agentic loop — harness built, not yet launched

  • tools/pick_target.py: picks the next function (or --cluster for a whole TU's worth at once), scoring toward YRpp-known functions in high-YRpp-coverage TUs with high call-graph degree. Status ("has this been touched?") is derived, not manually tracked — it diffs current src/ content against a freshly-regenerated pristine stub (tools/gen_stubs.pristine_stub_lines), so it can't drift out of sync the way a hand-maintained field could. First real pick: CRT::_delete (1790 callers) — high leverage, small, YRpp-known.
  • tools/export_function.py: Ghidra pseudo-C + disassembly for one address (already built during the Phase 0.5 smoke test).
  • CLAUDE.md: the concrete runbook — the loop steps, the hard rules learned the hard way (declare-in-class/define-out-of-line, never hardcode addresses, never invent names), and what's already handled so it isn't re-solved (flags, TU assignment caveats, status tracking).
  • tools/agentic_loop.py: spawns claude -p "<runbook prompt>" --dangerously-skip-permissions in a loop with a per-iteration timeout and short pause between runs, --max-iterations for a bounded trial. Not yet run continuously/unattended — a bigger commitment (autonomous commits, ongoing token spend) than the rest of the setup, worth a deliberate decision rather than defaulting to "on."

Phase 4 — Tracking and hygiene

  • tools/report.py publishes reccmp's aggregate report (per-TU and total %) on every commit; optionally wire into decomp.dev-style badge in README.
  • Nightly full reccmp run in CI (GitHub Actions, Windows runner with the portable VC6) — but never commit orig/gamemd.exe or any game asset to the repo.
  • Periodic human review passes over agent commits (enforce a "no analysis debris" rule).

5. Prioritization (what to decompile first)

  1. CRT/compiler-generated code: identify and fence off statically linked MSVC CRT — match it from CRT sources shipped with VC6, or exclude from scoring. Big % for free.
  2. Game-logic classes with dense YRpp coverage (TechnoClass, UnitClass, HouseClass, BulletClass…) — best signal for the AI agent, written independently from Ghidra output.
  3. Westwood shared libs without an external reference (INIClass, LCW, CRC, RNG, mixfile) — decompiled the same way as everything else, no shortcut source.
  4. Rendering/DirectDraw and networking last (least YRpp coverage, most gnarly).

6. Risks / open questions

  • Scale: ~5 MB EXE, 8,584 functions per Ghidra's analysis (measured, not guessed — see config/functions.csv), of which YRpp names 792 (~9%). Smaller than initially estimated; still a long project, but the phased skeleton means it's continuously useful (a growing annotated map) well before full match.
  • Exact compiler: if the Rich header reveals a nonstandard VC6 build/SP or Intel compiler for hot paths, flag matching gets harder — resolve in Phase 0 before scaling.
  • YRpp license unresolved (no LICENSE file) — mitigated by only ever extracting facts (addresses/offsets/names) into config/*.csv, never copying header text verbatim.
  • Steam binary vs community 1.001: Phobos.dll working in this install strongly implies address compatibility, but verify a handful of YRpp addresses in Ghidra in Phase 1.1.
  • Legal posture: user owns the game; repo is CC0-licensed reconstructed source only, written independently (not transcribed from GPL reference material); never ships game assets or the original EXE. Not being published yet, so no urgency, but the posture is decided up front so later work doesn't have to be re-litigated or re-licensed.

7. Immediate next steps

  1. Phase 0.1–0.2: hash + Rich header analysis of gamemd.exe.
  2. Stand up VC6 + reccmp smoke test.
  3. Write tools/yrpp_symbols.py and count how many functions YRpp actually names — that number sizes the whole project.