recipe(mpnet): add dell-research-harvard/lt-wikidata-comp-en (feature-extraction, sentence-similarity)

[DingmaomaoBJTU]

This draft adds curated WinML recipes for dell-research-harvard/lt-wikidata-comp-en, a LinkTransformer entity/record-linkage sentence embedder using mpnet / MPNetModel. It ships four recipe configs: feature-extraction and sentence-similarity, each in the catalog fp16 unquantized bucket plus w8a16 QDQ form. This is a recipe-only L0 contribution: the baseline auto-config on origin/main already builds MPNet through vendor support, so no per-architecture WinML code is changed. Highest verified goal is L1 PASS on CPU; GPU/NPU and full 10-bucket fp16 eval remain deferred/blocked as described below.

Summary

• Model: dell-research-harvard/lt-wikidata-comp-en.
• What it is: LinkTransformer entity/record-linkage sentence embedder fine-tuned from sentence-transformers/multi-qa-mpnet-base-dot-v1; Hugging Face model_type=mpnet, MPNetModel, encoder-only CLS-pooling.
• Added recipes: 4 configs total — feature-extraction + sentence-similarity, each with fp16 + w8a16 variants.
• Contribution type: recipe-only L0; baseline auto-config on origin/main already builds, so no WinML model code is included.

Effort / Goal / Outcome tiers

• Effort: L0 (new recipe, existing mpnet template/vendor path; no code).
• Goal ceiling: L1 (perf).
• Outcome: L0 (recipes + README).
• Target EPs: ["cpu"].

Files changed

• examples/recipes/README.md
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/feature-extraction_fp16_config.json
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/feature-extraction_w8a16_config.json
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/sentence-similarity_fp16_config.json
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/sentence-similarity_w8a16_config.json

models_all.json is intentionally untouched, matching prior recipe-PR precedent.

Recipe design

The recipes mirror the existing sentence-transformers/multi-qa-mpnet-base-dot-v1 template. The fp16 recipes use quant: null: this is the catalog unquantized bucket, realized per EP by --precision fp16; on CPU it remains FLOAT-dominant/fp32 by design. This is the catalog convention, not a mislabel (see #1097). The w8a16 recipes use QDQ UINT8/UINT16 quantization with quant.model_id set to dell-research-harvard/lt-wikidata-comp-en for calibration.

Verification / evidence

1. Recipe path(s)

• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/feature-extraction_fp16_config.json
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/feature-extraction_w8a16_config.json
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/sentence-similarity_fp16_config.json
• examples/recipes/dell-research-harvard_lt-wikidata-comp-en/sentence-similarity_w8a16_config.json

2. README row

Added rows:

Model	Task
dell-research-harvard/lt-wikidata-comp-en	feature-extraction
dell-research-harvard/lt-wikidata-comp-en	sentence-similarity

Updated total line to 77 curated (model, task) tuples and explicitly notes these new rows are build + CPU-perf verified, with full-bucket fp16 eval pending.

3. Build output dir

N/A — tester supplied build verdicts/timings and dtype counts, but did not supply a build output directory path.

4. Build log

L0 build PASS from tester verdicts:

Recipe	Verdict	Build time	Dtype evidence
feature-extraction fp16	PASS	193.4s	{FLOAT:200, INT32:1, INT64:55}
feature-extraction w8a16	PASS	176.5s	includes {UINT16:221, UINT8:194}
sentence-similarity fp16	PASS	184.2s	{FLOAT:200, INT32:1, INT64:55}

The fp16 dtype distribution is FLOAT-dominant with zero FLOAT16, which is expected unquantized-bucket-on-CPU behavior. The w8a16 dtype counts show genuine quantization.

5. Appended findings

Lane A reference only; not included in this Lane B PR:

• mpnet.json: mpnet-001, mpnet-002, mpnet-003.
• skill_meta/findings.json: _meta-061, _meta-062, _meta-063.

6. Optimum-coverage probe

VENDOR-ONLY. Baseline auto-config build on origin/main passed for MPNet; no WinML code change was needed.

7. Claimed (Effort, Goal, Outcome)

• Effort L0.
• Goal L1.
• Outcome L0.
• Ceiling held: L1 PASS.

8. Goal-ladder verdict table

Tier	Verdict	Evidence
L0 build	PASS	feature-extraction fp16 193.4s; feature-extraction w8a16 176.5s; sentence-similarity fp16 184.2s. fp16 dtypes {FLOAT:200, INT32:1, INT64:55}; w8a16 includes UINT16:221, UINT8:194.
L1 perf (CPU)	PASS	fp16 mean 1455.6ms (winml perf) / 2505.9ms (custom valid-input harness); w8a16 mean 7207.3ms (winml perf) / 5703.1ms (custom valid-input harness). w8a16 is slower than fp16 on CPU; expected quant inversion, not a defect.
GPU/NPU	HOST-BLOCKED	CPU-only host.

9. Methodology-evolution declaration

Learner recorded Lane A findings _meta-061, _meta-062, and _meta-063; these are intentionally not part of this Lane B model PR.

10. Perf & eval data

EP / Device	Precision	Verdict	Mean (winml perf)	Mean (custom valid-input harness)	Notes
CPU / cpu	fp16 bucket	PASS	1455.6ms	2505.9ms	CPU realizes this unquantized bucket as FLOAT-dominant/fp32 by design.
CPU / cpu	w8a16	PASS	7207.3ms	5703.1ms	Slower than fp16 on CPU; expected quant inversion. Value is size/NPU portability rather than CPU latency.
GPU / gpu	N/A	HOST-BLOCKED	—	—	CPU-only host; not tested.
QNN / npu	N/A	HOST-BLOCKED	—	—	CPU-only host; not runtime-tested. Static analyze reports QNN support as below.

No full fp16 eval matrix/task metric is claimed here. The README's all-10-bucket fp16-eval bar is not certified by this PR for the two new rows; that remains deferred to eval/CI.

11. Component / op-level data

winml analyze evidence from tester: 389 operators total, 17 unique operator types. QNN static support classification: 389/389 supported, 0 unsupported. This is an NPU-portability signal only; it is not runtime proof because NPU execution was HOST-BLOCKED on the CPU-only host. Analyze artifact path: N/A — tester did not supply a path.

12. Reproducible commands

N/A — tester did not supply exact reproducible command lines. Per role contract, this PR transcribes supplied verdicts and does not rerun build/perf/analyze/eval to generate missing report fields.

Coverage: partial

CPU is verified for build + perf through L1. GPU/NPU are HOST-BLOCKED on a CPU-only host and were not tested. Full fp16 eval across all 10 EP/device buckets is pending; this run's ceiling was L1 perf, not L2/L3 eval. QNN analyze showing 389/389 supported operators is a static portability signal only, not runtime certification.

Baseline

• Baseline branch/SHA: origin/main at 6cbc78fe.
• Tooling: winml 0.2.0.

[DingmaomaoBJTU]

VERDICT: APPROVE
coverage: partial
not_yet_tested_on: [gpu, npu, full-fp16-eval-matrix]
REQUEST_CHANGES: none

Independent reviewer: DingmaomaoBJTU. Posted as PR comment due same-identity approval constraint; did not use gh pr review.

Baseline freshness:

HEAD=1cc1caa38f026b87817c00282e959573fe9686da
origin_main=3f5e4683dfaf453c7874bae7cd300be0e07f5406
baseline_count=2
baseline_commit=6cbc78febbd62190881a44e52dd54feeaca996e1
6cbc78fe..origin/main: genai/perf-only files; no examples/recipes changes. Stale-baseline caveat noted; judged non-blocking for this pure-recipe PR.

Scope / no-leakage:

1cc1caa3 recipe(mpnet): add dell-research-harvard/lt-wikidata-comp-en (feature-extraction, sentence-similarity)
 examples/recipes/README.md                         |  4 +-
 .../feature-extraction_fp16_config.json            | 64 +++++++++++++++++
 .../feature-extraction_w8a16_config.json           | 82 ++++++++++++++++++++++
 .../sentence-similarity_fp16_config.json           | 64 +++++++++++++++++
 .../sentence-similarity_w8a16_config.json          | 82 ++++++++++++++++++++++
 5 files changed, 295 insertions(+), 1 deletion(-)

Files are exactly the 4 recipe JSONs plus examples/recipes/README.md. No src/, tests/, model_knowledge/, skill_meta/, or models_all.json. Commit trailer present: Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>.

Recipe JSON assertions:

target_files= ['feature-extraction_fp16_config.json', 'feature-extraction_w8a16_config.json', 'sentence-similarity_fp16_config.json', 'sentence-similarity_w8a16_config.json']
fp32_files= []
feature-extraction_fp16_config.json: loader.task=feature-extraction loader.model_type=mpnet loader.model_id=<ABSENT> quant_is_null=True mirror_allowed_only=True
feature-extraction_w8a16_config.json: loader.task=feature-extraction loader.model_type=mpnet loader.model_id=<ABSENT> quant.mode=qdq quant.model_id=dell-research-harvard/lt-wikidata-comp-en quant.weight_type=uint8 quant.activation_type=uint16 mirror_allowed_only=True
sentence-similarity_fp16_config.json: loader.task=sentence-similarity loader.model_type=mpnet loader.model_id=<ABSENT> quant_is_null=True mirror_allowed_only=True
sentence-similarity_w8a16_config.json: loader.task=sentence-similarity loader.model_type=mpnet loader.model_id=<ABSENT> quant.mode=qdq quant.model_id=dell-research-harvard/lt-wikidata-comp-en quant.weight_type=uint8 quant.activation_type=uint16 mirror_allowed_only=True
all_expected_present= True

README check:

17: Total: **77** (model, task) tuples with curated recipes. 75 pass fp16 eval on all 10 (EP, device) buckets; dell-research-harvard/lt-wikidata-comp-en (feature-extraction, sentence-similarity) ships fp16 + w8a16 mpnet recipes verified via build and CPU perf, with full-bucket fp16 eval pending.
43: | dell-research-harvard/lt-wikidata-comp-en | feature-extraction |
44: | dell-research-harvard/lt-wikidata-comp-en | sentence-similarity |

Rows are alphabetically placed between deepset/bert... and deepset/roberta..., and the Total line is honest (build+CPU-perf verified; full-bucket fp16 eval pending). No other table rows disturbed.

Independent recipe-driven build smoke:

# Exact requested shape with --use-cache + -o on this CLI:
Error: --output-dir and --use-cache are mutually exclusive.

# Re-ran recipe-driven without --use-cache; no --device/--ep/--precision overrides:
.\.venv\Scripts\winml.exe build -m dell-research-harvard/lt-wikidata-comp-en -c examples\recipes\dell-research-harvard_lt-wikidata-comp-en\feature-extraction_fp16_config.json -o temp\lt_review\fe_fp16 --rebuild
✅ Build complete in 145.6s
EXITCODE=0
MODEL_ONNX_PRESENT=1
BUILD_CONFIG_PRESENT=1
fe_fp16 initializer_dtype_names= {'FLOAT': 200, 'INT32': 1, 'INT64': 55}
fe_fp16 inputs=[('input_ids', [1, 512], 'INT32'), ('attention_mask', [1, 512], 'INT32')]
fe_fp16 outputs=[('last_hidden_state', [1, 512, 768], 'FLOAT')]

Optional w8a16 corroboration:

.\.venv\Scripts\winml.exe build -m dell-research-harvard/lt-wikidata-comp-en -c examples\recipes\dell-research-harvard_lt-wikidata-comp-en\feature-extraction_w8a16_config.json -o temp\lt_review\fe_w8a16 --rebuild
✅ Build complete in 218.0s
EXITCODE=0
initializer_dtype_names= {'FLOAT': 409, 'UINT8': 194, 'UINT16': 221, 'INT32': 195, 'INT64': 55}
build_config_quant.model_id=dell-research-harvard/lt-wikidata-comp-en; weight_type=uint8; activation_type=uint16

Coverage computation:
CPU build+perf is the reachable verified path from the tester table. GPU and NPU remain HOST-BLOCKED; full fp16 eval matrix remains pending. Therefore coverage is partial, with deferred set above. Static CI/checks observed green via gh pr checks (Analyze, CodeQL, lint, test shards, CLA all pass).

[DingmaomaoBJTU]

Update: recipes reorganized under cpu/cpu/ (tested EP/device layout)

Following the _meta-051 "one folder per EP+device" convention, the 4 recipes have been moved from the flat model dir into a nested cpu/cpu/ folder — reflecting the only EP/device combo actually validated (build + CPU perf). GPU/NPU were host-blocked, so no dml/gpu, qnn/npu, etc. folders are claimed.

examples/recipes/dell-research-harvard_lt-wikidata-comp-en/cpu/cpu/
  ├─ feature-extraction_fp16_config.json
  ├─ feature-extraction_w8a16_config.json
  ├─ sentence-similarity_fp16_config.json
  └─ sentence-similarity_w8a16_config.json

Nature of the change: pure git mv — 100% rename similarity, 0 insertions(+), 0 deletions(-). Recipe contents are byte-identical to the approved versions.

Re-verified: a recipe-driven fp16 build (winml build -c cpu/cpu/feature-extraction_fp16_config.json -m dell-research-harvard/lt-wikidata-comp-en, no override flags) still succeeds from the new path — export → optimize → model.onnx in ~104s.

Known caveat (deliberate): the e2e eval harness discover_recipe_variants (scripts/e2e_eval/utils/recipes.py:131) uses a flat, non-recursive glob (model_dir.glob("{task}_*_config*.json")), so recipes nested under <ep>/<device>/ are not auto-discovered until the harness is taught to recurse. This mirrors the real nested examples on main (facebook_dinov2-base, microsoft_swinv2-*), which keep flat defaults and add a nested qnn/npu/ override — here we intentionally went pure-nested per the requested layout. (Separately, this model's models_all.json entry is keyed to task: tabular-classification, not the authored feature-extraction/sentence-similarity, so the models_all-driven eval already keys it on a different task.)

[ssss141414]

Please revert readme. I will emphasize this on skill later.

[DingmaomaoBJTU]

@ssss141414 Done — reverted examples/recipes/README.md back to the base: count stays 75 and the two dell-research-harvard/lt-wikidata-comp-en rows are removed. Only the README is reverted; the recipe JSONs are unchanged. Will leave the README emphasis to the skill as you noted.