feat(depth-pro): support FP16 conversion and evaluation

[ssss141414]

Summary

Adds verified CPU recipes for apple/DepthPro-hf in FP32 and genuine FP16, and fixes two class-wide infrastructure defects exposed by the full Goal ladder:

• large external-data models no longer enter ORT's protobuf-size-unsafe in-memory shape-inference path during FP16 conversion;
• depth-estimation inference preserves optional field_of_view metadata required by DepthPro post-processing.

This replaces the intent of closed PR #943 against current main.

1. Recipe paths and auto-config diff

• examples/recipes/apple_DepthPro-hf/cpu/cpu/depth-estimation_fp32_config.json
• examples/recipes/apple_DepthPro-hf/cpu/cpu/depth-estimation_fp16_config.json

The exact-current-main auto-config (e95011d355d63e9309be1e4d314d582a2e2d2afe) and shipped FP32 recipe are semantically identical; the generated build config has only the runtime bookkeeping field "auto": false. The FP16 recipe is the same input/output contract plus quant.mode: "fp16" and fp16_keep_io_types: true.

examples/recipes/README.md is intentionally unchanged: it indexes production built-in models, and skill-produced recipes do not meet that publication contract.

2. Precision matrix

EP	Device	FP32	FP16	W8A8	W8A16
CPUExecutionProvider	CPU	PASS L0-L3	PASS L0-L3	N/A (not an NPU target)	N/A (not an NPU target)

Available runtime providers on the validation host were AzureExecutionProvider and CPUExecutionProvider. No GPU/NPU recipe or runtime claim is made without capable hardware. Static --ep all analysis was still run for both artifacts.

3. Build output directories

• Exact-main no-recipe baseline: temp/rerun943/baseline_e95011d3_build/
• FP32 recipe: temp/rerun943/recipe_fp32_build/
• FP16 recipe, fully fresh genuine-FP16 post-fix build: temp/rerun943/recipe_fp16_fresh_genuine/
• Earlier isolated FP16 conversion proof: temp/rerun943/recipe_fp16_retry/

4. Build logs

Exact-current-main baseline, with no recipe:

• source commit: e95011d355d63e9309be1e4d314d582a2e2d2afe
• winml 0.2.0
• ✅ Build complete in 588.4s
• export: 410.2s; optimize: 171.0s
• final external-data artifact: 3,803,028,992 bytes
• ONNX checker: PASS; input/output contract matches DepthPro

FP32 recipe:

• ✅ Build complete
• 1,312 FLOAT and 53 INT64 initializers
• external data: 3,803,028,992 bytes

FP16 recipe:

• initial run exported and optimized successfully, then reproduced google.protobuf.message.EncodeError in ORT FP16 shape inference;
• after the generalized fix, isolated conversion completed in 90.0s and the build completed with exit code 0;
• the fully fresh post-fix build from apple/DepthPro-hf is the final L0 proof: ✅ Build complete in 835.4s (export 537.9s, optimize 207.4s, FP16 84.2s);
• 1,309 FLOAT16, 3 FLOAT, and 53 INT64 initializers
• external data: 1,901,527,296 bytes

5. Findings

1. External tensor data does not protect an in-memory ModelProto from protobuf's serialization ceiling. ORT's default FP16 shape inference serializes the complete model, so convert_to_fp16() now sets disable_shape_infer for models at EXTERNAL_DATA_THRESHOLD scale. Export and optimization have already populated shapes.
2. WinMLModelForDepthEstimation previously discarded all outputs except predicted_depth. DepthPro's processor requires field_of_view; the new WinMLDepthEstimatorOutput preserves it while leaving it optional for single-output depth architectures.
3. A recipe-only classification would have been incorrect: L0/L1 looked healthy, but the genuine-FP16 and L3 steps exposed shared infrastructure defects.
4. Durable knowledge was appended in the adding-model-support skill repository as depth_pro-004 and depth_pro-005; it is intentionally not mixed into this product-code PR.

6. Optimum coverage probe

• HF model type: depth_pro
• task: depth-estimation
• vendor Optimum registration: no native depth_pro ONNX task
• after WinML registration: depth-estimation is available
• verdict: WINML-ONLY

The exporter already existed; this contribution does not add checkpoint-specific exporter branching.

7. Reissued charter

Reality contradicted the historical recipe-only charter, so the planner reissued it:

• Effort: L2 — two generalized shared-infrastructure fixes
• Goal ceiling: L3 — build, CPU perf/memory, PyTorch parity, and NYU Depth V2 task metrics
• Outcome: L1 — recipes + shared code + regression tests + durable findings; no new task family

8. Goal-ladder verdicts

Precision	Tier	Verdict	Evidence
FP32	L0	PASS	ONNX checker PASS; opset 17; expected input [1,3,1536,1536]; both outputs present; 3,803,028,992-byte external data co-located
FP32	L1	PASS	CPU p50 76,868.743ms; total RSS delta 9,984.25MB; one measured iteration, no warmup
FP32	L2	PASS	predicted_depth: relative L2 7.224452521572391e-07, cosine 0.999999999999883; field_of_view: relative L2 7.430005631944269e-07
FP32	L3	PASS	NYU Depth V2 sample: abs_rel 0.04516238443198268, RMSE 0.18053944191186216, delta1 0.9685807291666667, 307,200 valid pixels
FP16	L0	PASS	ONNX checker PASS; 1,309 FLOAT16 initializers; 1,901,527,296-byte external data co-located; FP32 I/O retained
FP16	L1	PASS	CPU p50 72,293.889ms; total RSS delta 10,569.51MB; one measured iteration, no warmup
FP16	L2	PASS	predicted_depth: relative L2 0.00023954855291130023, cosine 0.9999999763468135; field_of_view: relative L2 4.861346542043536e-05
FP16	L3	PASS	NYU Depth V2 sample: abs_rel 0.0451814304400756, RMSE 0.1805982524308017, delta1 0.96857421875, 307,200 valid pixels

The refreshed no-recipe FP32 baseline also passed build and one CPU perf iteration at p50 45,519.223ms with total RSS delta 10,132.10MB. Single-iteration CPU latency is noisy and this PR does not claim an FP32 speedup; its measurable deltas are genuine FP16 support and working task post-processing.

9. Methodology declaration

Methodology friction was observed: the historical recipe-only/L1 framing stopped before genuine precision and task-level validation. The active skill now requires broad precision attempts, a complete L0-L3 march, recipe-vs-auto-config reducibility, nested EP/device paths, all-EP static analysis, and no production README edit. Those methodology changes live on the skill's Lane A commit 8bb9b419a5e58ad90ccf2a1b6165c44ce6e3d757, not in this model PR.

10. Perf and evaluation table

Artifact	CPU p50 (1 iter)	RSS total delta	abs_rel	RMSE	delta1
exact-main baseline FP32	45,519.223ms	10,132.10MB	inherited / not re-scored	inherited / not re-scored	inherited / not re-scored
recipe FP32	76,868.743ms	9,984.25MB	0.0451623844	0.1805394419	0.9685807292
recipe FP16	72,293.889ms	10,569.51MB	0.0451814304	0.1805982524	0.9685742188

Dataset methodology: sayakpaul/nyu_depth_v2, validation, revision refs/convert/parquet, streaming, no shuffle, one deterministic sample, image column image, depth column depth_map, depth kind depth.

11. Operator analysis

winml analyze --ep all completed for both artifacts:

• FP32: 3,109 operators, 19 unique types
• FP16: 3,112 operators, 20 unique types (three boundary Casts)
• OpenVINO CPU rule data: 18/19 known op types classified supported, zero partial, zero unsupported; Resize and Pad remain unknown
• runtime_support: false means this was static rule coverage, not an OpenVINO runtime claim

Top counts are unchanged apart from FP16 boundary Casts: Add 666, MatMul 577, Mul 432, Transpose 382, Reshape 303, Slice 259, LayerNormalization 147.

12. Reproduction commands

# Exact-main baseline (no recipe)
winml build -m apple/DepthPro-hf -o temp/rerun943/baseline_e95011d3_build --ep cpu --device cpu --no-analyze --no-compile --rebuild

# Recipe builds
winml build -c examples/recipes/apple_DepthPro-hf/cpu/cpu/depth-estimation_fp32_config.json -m apple/DepthPro-hf -o temp/rerun943/recipe_fp32_build --ep cpu --device cpu --no-analyze --no-compile --rebuild
winml build -c examples/recipes/apple_DepthPro-hf/cpu/cpu/depth-estimation_fp16_config.json -m apple/DepthPro-hf -o temp/rerun943/recipe_fp16_fresh_genuine --ep cpu --device cpu --precision fp16 --no-analyze --no-compile --rebuild

# Static analysis
winml analyze --model temp/rerun943/recipe_fp32_build/model.onnx --ep all --output temp/rerun943/analyze_fp32_all.json --overwrite --format json --no-color
winml analyze --model temp/rerun943/recipe_fp16_fresh_genuine/model.onnx --ep all --output temp/rerun943/analyze_fp16_all.json --overwrite --format json --no-color

# CPU perf
winml perf --model temp/rerun943/recipe_fp32_build/model.onnx --task depth-estimation --device cpu --ep cpu --precision fp32 --iterations 1 --warmup 0 --output temp/rerun943/recipe_fp32_perf.json --overwrite --skip-build --format json --no-color
winml perf --model temp/rerun943/recipe_fp16_fresh_genuine/model.onnx --task depth-estimation --device cpu --ep cpu --precision fp16 --iterations 1 --warmup 0 --output temp/rerun943/recipe_fp16_perf.json --overwrite --skip-build --format json --no-color

# L3 task metric (repeat once per artifact/precision)
winml eval --model temp/rerun943/recipe_fp32_build/model.onnx --model-id apple/DepthPro-hf --task depth-estimation --device cpu --ep cpu --dataset sayakpaul/nyu_depth_v2 --dataset-revision refs/convert/parquet --split validation --samples 1 --no-shuffle --streaming --column input_column=image --column depth_column=depth_map --column depth_kind=depth --output temp/rerun943/l3_fp32_depth_eval_fixed.json --overwrite --skip-build --format json --no-color

# Focused regression suite
python -m pytest tests/unit/models/winml/test_depth_estimation.py tests/unit/test_quant_passes.py -q
python -m ruff check src/winml/modelkit/models/winml/depth_estimation.py src/winml/modelkit/quant/fp16.py tests/unit/models/winml/test_depth_estimation.py tests/unit/test_quant_passes.py

Post-rebase checks: 36 passed; Ruff: All checks passed.

[ssss141414]

Independent reviewer verdict: APPROVE

Reviewed PR head 85529d8173f0430ba776044f55dc4e55f94f6251 against base e95011d355d63e9309be1e4d314d582a2e2d2afe.

Scope and engineering review

• Diff contains exactly six intended files: two nested CPU recipes, two shared source files, and two focused test files.
• examples/recipes/README.md is untouched.
• convert_to_fp16() uses the existing EXTERNAL_DATA_THRESHOLD and serialization-safe get_model_size() abstraction rather than a model-id branch. Skipping redundant ORT shape inference only for external-data-scale models is a safe class-wide fix; small-model behavior is tested unchanged.
• WinMLDepthEstimatorOutput remains a DepthEstimatorOutput; optional field_of_view preserves DepthPro metadata without changing single-output depth models. Tests cover attribute/dict access and absence semantics.
• Recipes match the 1536×1536 DepthPro contract. The FP16 recipe is genuine only when the explicit --precision fp16 build path is selected; the emitted artifact proves 1,309 FLOAT16 initializers and a 1,901,527,296-byte external-data shard.

Commands independently re-run

• Focused pytest: 36 passed in 24.44s.
• Ruff on all four changed Python files: All checks passed.
• Exact-main no-recipe baseline rebuilt from e95011d3: Build complete in 588.4s; ONNX checker passed.
• Fully fresh recipe/model FP16 build with explicit precision: Build complete in 835.4s (export 537.9s, optimize 207.4s, FP16 84.2s).
• Fresh FP16 structural check: ONNX checker passed; 1,309 FLOAT16 / 3 FLOAT / 53 INT64 initializers; expected two outputs present.

Evidence audit

• CPU FP32/FP16 L0–L3 rows are internally consistent and carry structural, latency/memory, seeded PyTorch-parity, and NYU Depth V2 metrics.
• winml analyze --ep all evidence is correctly represented as static rule coverage rather than fabricated runtime coverage.
• Unavailable GPU/NPU tuples are not claimed.

No blocking correctness, maintainability, scope, or evidence issue found. GitHub may reject a formal approval from the PR author's identity; this comment records the structured reviewer verdict instead.