Skip to content

README: point paper readers at the v1.0-iccv2025 release - #106

Open
jonfroehlich wants to merge 2 commits into
mainfrom
docs/paper-version-pointer
Open

README: point paper readers at the v1.0-iccv2025 release#106
jonfroehlich wants to merge 2 commits into
mainfrom
docs/paper-version-pointer

Conversation

@jonfroehlich

@jonfroehlich jonfroehlich commented Aug 3, 2026

Copy link
Copy Markdown
Member

main has moved well past the ICCV'25 paper (post-paper analyses, benchmarks, RampNet 2.0 groundwork), and it will keep moving. The frozen paper artifact already exists — the v1.0-iccv2025 tag and its "paper state" GitHub Release — but nothing on the README told a paper reader that, or where to find it. This adds a callout at the top pointing at the frozen releases, and notes that the published HF model/dataset are versioned independently and match the paper.

Two pointers, not one (added in code review, 4190e4c): the note originally sent a paper reader to v1.0-iccv2025 alone — which is exactly the tag whose evaluators the Erratum fifteen lines below says bias precision and recall upward. It now names both v1.0-iccv2025 (reproduce the paper as written) and v1.1-corrected-eval (same code and weights, standard one-to-one matching), and links the Erratum from the note itself.

⚠️ Merge ordering: v1.1-corrected-eval does not exist yet. Do not merge until the tag and its release are created (proposed anchor ee42044, 2026-07-24 — see the review comment for why), or main's README ships a 404.

Decided with Jon (2026-08-03): RampNet2-era work stays in this repo with the tag as the paper boundary; a repo split is only revisited if a genuinely new pipeline stops importing this code.

🤖 Generated with Claude Code (claude-fable-5)

jonfroehlich and others added 2 commits August 3, 2026 15:52
main keeps evolving past the paper; the tag (and its GitHub Release,
"paper state") is where the as-published version lives, and the HF model
and dataset are versioned independently. One callout at the top so a
reader landing on an evolving main finds the frozen artifact in one click.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The note steered a paper reader at v1.0-iccv2025 — which is exactly the tag
whose evaluators the Erratum fifteen lines below says bias precision and recall
upward. Someone who reads the note and clicks through never sees that.

Two pointers instead of one, and the Erratum linked from the note: paper state
for reproducing the paper as written, v1.1-corrected-eval for the same code and
weights scored under standard one-to-one matching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jonfroehlich

Copy link
Copy Markdown
Member Author

Code review — and one change, in 4190e4c

Verified: the tag and release both exist and are published — v1.0-iccv20258783157, release "RampNet v1.0 — ICCV'25 CV4A11y (paper state)", isDraft: false, published 2026-07-02. Both Hugging Face links resolve. The alert sits at top level after a closed </p> with a blank line before it, so GitHub parses it as a [!NOTE] rather than swallowing it into the surrounding raw HTML.

The "published model … matches the paper" claim is accurate: scripts/export_hf_model.py's own docstring records the re-publish as "wrapper + corrected card; no retrain", so the weights are the paper's.

The one substantive finding

The note steered a paper reader at v1.0-iccv2025 for "RampNet exactly as the ICCV'25 paper describes it" — which is precisely the tag whose evaluators the Erratum, fifteen lines below, says bias precision and recall upward. Someone who reads the note and clicks through never sees that. Given the standing rule that caveats travel with the numbers, that's a gap worth closing in the note itself rather than trusting the reader to scroll.

Changed in 4190e4c: two pointers instead of one, with the Erratum linked from the note — v1.0-iccv2025 for reproducing the paper as written, v1.1-corrected-eval for the same code and weights scored under standard one-to-one matching.

⚠️ Merge ordering — this PR now depends on a tag that does not exist yet

v1.1-corrected-eval is proposed, not created. Do not merge this until the tag and its release exist, or main's README ships a 404.

The proposed anchor is ee42044 ("Route the benchmark scorer through the shared matcher (#42)", 2026-07-24), and the draft release notes are with Jon. Three reasons for that commit rather than main or the earlier a9ed8a5:

  • The README Erratum text is byte-identical from ee42044 to current main — the only README changes since are the "Running the Tests" section — so everything the release would claim is exactly true at that SHA.
  • The corrected Stage 2 table still reproduces there. ee42044 refactored match_predictions() to delegate to greedy_match(), and the refactor is behaviour-preserving: same confidence-descending sort, same nearest-unclaimed-within-radius rule, with saw_in_range as added bookkeeping.
  • It is the first commit where the Stage 1 half of the story is also true — both evaluators unified onto one matching core, and the README saying honestly that the Stage 1 agreement precision has not been re-run.

For the record, the fix is entirely after the paper tag, by about a year:

commit date
v1.0-iccv2025 points at 8783157 2025-08-15
Corrected matcher + rampnet/metrics.py ed9ec43 2026-07-01
Corrected gold-set results committed a9ed8a5 2026-07-02
README Erratum added 6d48619 2026-07-02
Stage 1 redundant points → FPs (#33) ed75586 2026-07-23
Benchmark scorer onto the shared matcher (#42) ee42044 2026-07-24

One loose end for the release notes, not for this PR

The corrected numbers are a property of (code SHA, weights revision, gold set), not of the code alone — and the Hugging Face model revision is not recorded anywhere in this repo. Worth naming it in the release notes so the triple is stated in one place. Related: if the paper-state weights are tagged v1.0-paper on the Hub, the same frozen artifact carries two different names in two places; worth cross-referencing in both the release notes and the model card. (Not verified from here — check the Hub side before writing it down.)

🤖 Generated with Claude Code (claude-opus-5[1m])

@jonfroehlich

Copy link
Copy Markdown
Member Author

Follow-up: the Hugging Face side, now verified

My earlier comment flagged the Hub naming question as unverified. Checked it against the Hub API — here is what is actually true, so nothing goes into a public release note on memory.

The refs

repo branches tags
projectsidewalk/rampnet-model main = 606a1195 (2026-07-24) v1.0-paper = 1078bcd6
projectsidewalk/rampnet-dataset main = ee882e3f (2025-08-15) none

So the naming mismatch is real: the frozen paper artifact is v1.0-iccv2025 on GitHub and v1.0-paper on the Hub. Worth cross-referencing in both release notes and the model card rather than renaming anything.

The dataset claim in this PR's note is trivially safe — a single main ref, untouched since 2025-08-15, so it is paper state by construction.

"No retrain" is now proven, not just documented. model.safetensors is not byte-identical across the two model revisions, which is worth knowing before anyone quotes a file hash. The difference is entirely packaging:

  • 380 tensors on both, same dtypes, same shapes, same byte offsets
  • file size delta = 2,312 bytes = 380 keys × 6 chars of model. prefix (2,280) + 32 bytes of {"format": "pt"} metadata — the delta is fully accounted for
  • the 360,202,244-byte tensor payload hashes identically on both:
sha256(tensor payload) = ea8c148eee227b32bf69ab2a2da0bc937b28405e46397d6cf38f1397e512132a

One correction to what I said earlier. I suggested naming the Hub revision in the v1.1-corrected-eval notes; the right one to name is v1.0-paper (1078bcd6), not main. The corrected gold-set numbers were committed 2026-07-02 (a9ed8a5), three weeks before the model re-publish on 2026-07-24 — so they were measured against the weights that tag now points at. main reproduces them too, since the payload is bit-identical, but naming it would misdescribe what was actually run. The draft release notes have been updated accordingly.

This also closes the loose end from my earlier comment: the Hub revision was recorded nowhere in this repo. The payload sha256 above is the content hash that makes the weights checkable without trusting a ref name, which is the discipline the rest of the replication story already uses.

🤖 Generated with Claude Code (claude-opus-5[1m])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant