Skip to content

feat(jacobian_lens): add artifact registry + short-name resolution in from_pretrained#1537

Merged
jlarson4 merged 3 commits into
TransformerLensOrg:devfrom
mukund1985:feat/jacobian-lens-registry
Jul 24, 2026
Merged

feat(jacobian_lens): add artifact registry + short-name resolution in from_pretrained#1537
jlarson4 merged 3 commits into
TransformerLensOrg:devfrom
mukund1985:feat/jacobian-lens-registry

Conversation

@mukund1985

Copy link
Copy Markdown
Contributor

Closes #1536.

What

Adds jacobian_lens_registry.json bundling 37 pre-fitted J-lens artifacts from neuronpedia/jacobian-lens, and teaches JacobianLens.from_pretrained to resolve short model names and Hugging Face model IDs automatically.

Before (users had to know the full Hub subpath):

JacobianLens.from_pretrained(
    "neuronpedia/jacobian-lens",
    filename="gemma-2-2b/jlens/Salesforce-wikitext/gemma-2-2b_jacobian_lens.pt",
)

After (both of these work):

JacobianLens.from_pretrained("gemma-2-2b")
JacobianLens.from_pretrained("google/gemma-2-2b")

Resolution order

  1. Local .pt file — unchanged
  2. Local directory — unchanged
  3. Registry short name or HF model ID alias — resolved to neuronpedia/jacobian-lens + exact subpath
  4. Explicit Hub repo + filename kwarg — unchanged (fully backward-compatible)

Registry coverage (37 models)

  • Gemma 2 (2b / 9b / 27b, base + instruct)
  • Gemma 3 (270m / 1b / 4b / 12b / 27b, base + instruct)
  • Gemma 4 (31b / e2b / e4b)
  • GPT-2 Small (aliases: gpt2, openai-community/gpt2), GPT-OSS 20B
  • Llama 3.1 (8b base + instruct), Llama 3.3 (70b instruct)
  • OLMo 3 (7b / 32b), Pythia 70M-deduped
  • Qwen 2.5 (7b-it), Qwen 3 (1.7b / 4b / 8b / 14b / 32b), Qwen 3.5 (0.8b / 2b / 4b / 9b / 27b)

The registry JSON is generated from neuronpedia/jacobian-lens sibling list and includes the exact filenames (including the naming irregularities in Gemma 3 base -pt suffix and Gemma 4 uppercase size suffix).

Tests

9 new unit tests in TestRegistry:

  • Registry JSON validity and required-field checks
  • _resolve_registry_entry by short name and by HF model ID alias
  • Unknown model returns None (passthrough to explicit Hub path)
  • from_pretrained short-name resolution end-to-end
  • from_pretrained HF alias → same artifact as short name
  • Unknown name forwards repo_id + filename kwargs unchanged (backward compat)
  • Registry filename wins over explicit filename kwarg when name resolves
  • GPT-2's three entry points (gpt2-small / gpt2 / openai-community/gpt2) all resolve to the same file

… from_pretrained

Adds jacobian_lens_registry.json bundling 37 pre-fitted J-lens checkpoints
from neuronpedia/jacobian-lens, and updates JacobianLens.from_pretrained to
resolve short model names and HF model IDs automatically.

Before this change users had to remember the full Hub subpath:

    JacobianLens.from_pretrained(
        'neuronpedia/jacobian-lens',
        filename='gemma-2-2b/jlens/Salesforce-wikitext/gemma-2-2b_jacobian_lens.pt',
    )

After this change either of these works:

    JacobianLens.from_pretrained('gemma-2-2b')
    JacobianLens.from_pretrained('google/gemma-2-2b')

Resolution order:
1. Local file path  (unchanged)
2. Local directory  (unchanged)
3. Registry short name or HF model ID alias → resolves repo_id + filename
4. Explicit Hub repo + filename kwarg  (unchanged, fully backward-compatible)

Registry coverage (37 models as of 2026-07-23):
- Gemma 2 (2b/9b/27b, base + instruct)
- Gemma 3 (270m/1b/4b/12b/27b, base + instruct)
- Gemma 4 (31b/e2b/e4b)
- GPT-2 Small, GPT-OSS 20B
- Llama 3.1 (8b base + instruct), Llama 3.3 (70b instruct)
- OLMo 3 (7b / 32b), Pythia 70M-deduped
- Qwen 2.5 (7b-it), Qwen 3 (1.7b/4b/8b/14b/32b), Qwen 3.5 (0.8b/2b/4b/9b/27b)

Tests: 9 new unit tests covering short-name lookup, HF-ID alias lookup,
unknown-model passthrough, filename override behaviour, and GPT-2's three
aliased entry points.

Closes TransformerLensOrg#1536.
@mukund1985

Copy link
Copy Markdown
Contributor Author

Hey @jlarson4 — happy to have a review when you get a chance. Let me know if you want the registry to carry any additional metadata (n_prompts, dtype) or if the alias coverage needs adjusting for any of the models.

@mukund1985 mukund1985 closed this Jul 23, 2026
@mukund1985 mukund1985 reopened this Jul 23, 2026
@jlarson4

Copy link
Copy Markdown
Collaborator

This looks good to my eyes @mukund1985, I will let CI finish its thing and then merge

@mukund1985

Copy link
Copy Markdown
Contributor Author

@jlarson4 CI is all green — 22 checks passed, ready to merge whenever you are.

@jlarson4
jlarson4 merged commit 08d7ff2 into TransformerLensOrg:dev Jul 24, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants