Skip to content

recipe(cross-encoder/ms-marco-MiniLM-L4-v2): add text-classification fp32/fp16 recipes#1121

Open
codykk wants to merge 1 commit into
mainfrom
yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe
Open

recipe(cross-encoder/ms-marco-MiniLM-L4-v2): add text-classification fp32/fp16 recipes#1121
codykk wants to merge 1 commit into
mainfrom
yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe

Conversation

@codykk

@codykk codykk commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Recipe-only contribution (Effort L0). Adds fp32 and fp16 recipe configs for
cross-encoder/ms-marco-MiniLM-L4-v2 (BertForSequenceClassification, 4 layers, task text-classification)
on two verified EP/device combinations: CPU and DML (GPU). Goal L1 (perf) PASS on all 4 configs.


1. Recipe path(s)

  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp16_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp16_config.json

2. README row

N/A — README not modified in this PR.

3. Build output dir

  • temp/minilm4_cpu_fp32/
  • temp/minilm4_cpu_fp16/
  • temp/minilm4_dml_fp32/
  • temp/minilm4_dml_fp16/

4. Build log

  • cpu/cpu fp32: ✅ Build complete in 16.2s (Export 3.8s, Optimize 12.0s)
  • cpu/cpu fp16: ✅ Build complete in 16.9s (Export 3.5s, Optimize 12.2s, FP16 0.7s)
  • dml/gpu fp32: ✅ Build complete in 16.5s (Export 3.7s, Optimize 12.3s)
  • dml/gpu fp16: ✅ Build complete in 17.3s (Export 3.8s, Optimize 12.5s, FP16 0.7s)

5. Appended findings

N/A — recipe-only L0 contribution.

6. Optimum-coverage probe

bert architecture fully supported by Optimum and winml. winml inspect confirms all components at "Default" status.

7. Claimed (Effort, Goal, Outcome)

Axis Tier
Effort L0 (recipe-only)
Goal L1 (build + perf)
Outcome L0 (recipe + report)

8. Goal-ladder verdict table

Tier Verdict Evidence
L0 (build) PASS All 4 configs build successfully
L1 (perf) PASS All 4 configs produce valid latency/throughput (see item 10)
L2 N/A Goal ceiling = L1
L3 N/A Goal ceiling = L1

9. Methodology-evolution declaration

No methodology friction observed.

10. Perf & eval data

EP / Device Precision Verdict Throughput RAM Δ VRAM Δ (local)
OpenVINOExecutionProvider / cpu fp32 PASS 82.69 samples/s +159.30 MB
OpenVINOExecutionProvider / cpu fp16 PASS 85.10 samples/s +160.10 MB
DmlExecutionProvider / gpu fp32 PASS 81.25 samples/s +283.00 MB +171.30 MB
DmlExecutionProvider / gpu fp16 PASS 31.42 samples/s +243.20 MB +101.00 MB
QNNExecutionProvider / npu * HOST-BLOCKED No NPU hardware

11. Component / op-level data

  • Architecture: 4 hidden layers, 12 attention heads, hidden size 384
  • Fusion patterns applied: gelu_fusion, matmul_add_fusion

12. Reproducible commands

# Build (cpu/cpu fp32)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp32_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_cpu_fp32

# Build (cpu/cpu fp16)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_cpu_fp16 --precision fp16

# Build (dml/gpu fp32)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp32_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_dml_fp32 --ep dml --device gpu

# Build (dml/gpu fp16)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_dml_fp16 --ep dml --device gpu --precision fp16

# Perf
winml perf -m temp/minilm4_cpu_fp32/model.onnx --device cpu --iterations 20
winml perf -m temp/minilm4_cpu_fp16/model.onnx --device cpu --iterations 20
winml perf -m temp/minilm4_dml_fp32/model.onnx --device gpu --ep dml --iterations 20
winml perf -m temp/minilm4_dml_fp16/model.onnx --device gpu --ep dml --iterations 20

@codykk codykk marked this pull request as ready for review July 16, 2026 04:05
@codykk codykk requested a review from a team as a code owner July 16, 2026 04:05
…MiniLM-L4-v2

Recipe-only contribution. Adds fp32 and fp16 recipe configs for
cross-encoder/ms-marco-MiniLM-L4-v2 (BertForSequenceClassification, 4 layers)
on two verified EP/device combinations: cpu/cpu and dml/gpu.

Goal ladder (verified on local hardware):
- L0 (build): PASS on all 4 configs
- L1 (perf):
  - cpu/cpu fp32: 82.69 samples/s
  - cpu/cpu fp16: 85.10 samples/s
  - dml/gpu fp32: 81.25 samples/s
  - dml/gpu fp16: 31.42 samples/s
@codykk codykk force-pushed the yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe branch from 3f3bf46 to 3bb2f8b Compare July 16, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant