Skip to content

Add config tests for DeepSeek V3 architecture adapter#1531

Merged
jlarson4 merged 1 commit into
TransformerLensOrg:devfrom
SanjidMzi:test/deepseek-v3-adapter-config
Jul 22, 2026
Merged

Add config tests for DeepSeek V3 architecture adapter#1531
jlarson4 merged 1 commit into
TransformerLensOrg:devfrom
SanjidMzi:test/deepseek-v3-adapter-config

Conversation

@SanjidMzi

@SanjidMzi SanjidMzi commented Jul 22, 2026

Copy link
Copy Markdown

Part of #1302.

This pr adds unit tests for DeepSeekV3ArchitectureAdapter covering the config flags it
sets, the component-mapping structure and HF module paths, the MLA attention
submodule mapping, and the (empty) weight-conversion set.

V3 differs from V2 at the unit level: V3 always compresses Q via
the two-stage LoRA path (that's why there is no direct q_proj fallback and every
attention projection is required), q_a_layernorm is an RMSNorm rather than a
plain component, and the router gate is bridged as an optional component.

Behavioural coverage (forward pass, hook firing) already lives in
tests/integration/model_bridge/test_deepseek_adapter.py; these tests
complement it at the unit tier, following the same pattern as the existing
test_deepseek_v2_adapter.py.

make format, uv run mypy ., and make unit-test all pass locally.

@jlarson4

Copy link
Copy Markdown
Collaborator

Looks great! Thank you for putting this together @SanjidMzi!

@jlarson4
jlarson4 merged commit 0d04eec into TransformerLensOrg:dev Jul 22, 2026
49 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants