recipe(bert): add dell-research-harvard/lt-un-data-fine-fine-es configs

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Summary

• Adds CPU fp32/fp16 recipe configs for dell-research-harvard/lt-un-data-fine-fine-es.
• Covers feature-extraction and sentence-similarity tasks.
• Refreshes the configs with eager transformers attention compatibility.

Validation

• Split cleanup only; no recipe validation was run in this pass.

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

APPROVE (independent reviewer; comment-only approval because same GitHub identity cannot use gh pr review --approve)

Coverage: full for the declared CPU target (CPUExecutionProvider / cpu); deferred EPs: none for this CPU-only PR. Note: the PR body says "coverage partial" due GPU/NPU host-unreachable, but its own target EP set is [cpu], so I treat the verified coverage as full for the shipped scope.

Evidence summary:

1. Diff scope / base

• PR recipe(bert): add dell-research-harvard/lt-un-data-fine-fine-es configs #1185 head 37697b340d470f8ecbc496a1c6ff822bef172cb5; base main at d9feefffccecfe47f08b7d2cfffb11fe57eb94df; detached worktree merge-base = d9feefffccecfe47f08b7d2cfffb11fe57eb94df.
• git rev-list --count HEAD..origin/main = 0 (not behind).
• gh api .../pulls/1185/files --jq '.[].filename' and local git diff --name-only origin/main...HEAD both show exactly these four files and nothing else:
  • examples/recipes/dell-research-harvard_lt-un-data-fine-fine-es/cpu/cpu/feature-extraction_fp16_config.json
  • examples/recipes/dell-research-harvard_lt-un-data-fine-fine-es/cpu/cpu/feature-extraction_fp32_config.json
  • examples/recipes/dell-research-harvard_lt-un-data-fine-fine-es/cpu/cpu/sentence-similarity_fp16_config.json
  • examples/recipes/dell-research-harvard_lt-un-data-fine-fine-es/cpu/cpu/sentence-similarity_fp32_config.json

2. Recipe correctness

• All four recipe JSONs parse successfully.
• Confirmed: opset 17, batch 1, inputs input_ids, attention_mask, token_type_ids all int32 shape [1,512]; value ranges [0,31002], [0,2], [0,2]; output last_hidden_state; optim.clamp_constant_values=true; quant:null; compile:null; loader model_class=AutoModel, model_type=bert; task matches filename (feature-extraction or sentence-similarity).
• Blob OIDs from git ls-files -s:
  • FE fp16/fp32 both 659160eba98de1ec86bdb70a430b5927f2f91a3d
  • SS fp16/fp32 both 6c1a31273ef2d5a1c50683f4d5de26cde1e7e9b5
• git diff --no-index confirms each fp16/fp32 pair is byte-identical.

3. Baseline gate (central honesty claim)

• Ran no-recipe baseline: winml build -m dell-research-harvard/lt-un-data-fine-fine-es -o C:\Users\qiowu\source\repos\wcli-review-ltun\rev_baseline.
• Parsed stdout: ✅ Build complete in 112.4s.
• Quantize stage present: ✅ Quantize 40.4s, precision uint8/uint16 (weight/activation).
• Built artifact is w8a16: winml_build_config.json has weight_type: uint8, activation_type: uint16; final external-data artifact totals ~127.8 MB (model.onnx 0.4 MB + model.onnx.data 127.4 MB).
• Therefore the PR's L0-not-L0★ framing is correct: baseline succeeds; recipe does not fix a crash.

4. Recipe L0 builds and ONNX structure

• FE fp32 recipe build: ✅ Build complete in 92.9s; no Quantize stage; winml_build_config.json quant = null; model.onnx.data co-located (416.79 MB).
• SS fp32 recipe build: ✅ Build complete in 94.7s; no Quantize stage; winml_build_config.json quant = null; model.onnx.data co-located (416.79 MB).
• ONNX validation with onnx.load(..., load_external_data=False):
  • FE: IR 8, ai.onnx opset 17, inputs input_ids/attention_mask/token_type_ids elem_type 6 shape [1,512], output last_hidden_state elem_type 1 shape [1,512,768].
  • SS: IR 8, ai.onnx opset 17, same input/output names, dtypes, and shapes.

5. Independent L2 parity

• I wrote and ran my own script (rev_l2_parity.py) against the freshly-built FE model.onnx, not the producer's temp script.
• Sentences: 5 Spanish sentences, padded/truncated to max_length 512, tokenizer-provided token_type_ids, ONNX inputs cast to int32, ORT CPUExecutionProvider.
• Raw last_hidden_state max-abs per sentence:
  • 9.357929e-05
  • 3.908873e-04
  • 3.681183e-04
  • 5.857944e-04
  • 9.212494e-04
• Overall max-abs: 9.212494e-04 (< 1e-3).
• Mean-pooled + L2-normalized paired embedding cosine: [1.0, 1.0, 1.0, 0.9999999404, 1.0]; min = 0.9999999404 (>= 0.9999).
• PyTorch-vs-ONNX retrieval ranking identical: true.

6. L1 perf / op-level body audit

• Re-ran winml perf -m rev_fe\model.onnx --device cpu --ep cpu: avg 686.55 ms, P50 654.23 ms, min 408.95 ms, fp32, CPUExecutionProvider. This is within ±20% of the PR body's 623.22 ms / 609.55 ms / 444.80 ms.
• Re-ran winml analyze -m rev_fe\model.onnx: 384 total operators, 15 unique types, QNNExecutionProvider(NPU) 384/0/0/0, Ready to deploy — matches the PR body's component claim.

7. Optimum coverage probe

• Re-ran probe for bert:
  • vendor tasks: feature-extraction, fill-mask, multiple-choice, question-answering, text-classification, token-classification
  • after ensure_hf_models_registered(): same list
  • added_by_winml = []
• Result: VENDOR-ONLY, matching PR body item 6 and supporting Effort L0.

8. PR body audit

• Tiers are stated as Effort L0 · Goal ceiling L2 · Outcome L0, not L0★.
• Body explicitly says baseline succeeds and emits w8a16; it does not claim to fix a crash.
• Item 9 discloses the L0★ → L0 re-tiering.
• README row omission is explicitly justified as avoiding overclaim for the all-EP README table.
• Numeric claims checked against independent reruns are either matched or within expected tolerance; my independent L2 numbers pass the stated thresholds.

Concerns: no blocking concerns. Only non-blocking wording note: item 10's "coverage partial" is confusing because the declared target EPs are [cpu]; for this PR's CPU-only shipped scope, coverage is full with no deferred EPs.

[ssss141414]

ACCEPTABLE

Reviewed exact head 86821d476bcfcc487dc993783557e49b0a930dfb against freshly fetched base c45e9513964bc0bc55ad9cdd612537e96349c8a3 as independent reviewer ssss141414.

The four-file L0 recipe scope remains clean and unchanged. All previously passed candidate gates remain applicable at the stable head/base: reviewer-owned fp16 build and inference, all four L0/L1/L2 rows, the bounded L3 functional smoke, Analyze evidence, Ruff, mypy, 8,043 non-hardware tests, and all nine exact-head GitHub checks.

Both prior blockers are repaired. Direct inspection of all four sealed HTP files and Lane A commit 65b6d5f6d3857dff0af5891685a269b3ca36d9c1 confirms the complete root hierarchy: BertEmbeddings order 1, BertEncoder order 2, and BertPooler order 87. The SHA-verified cross-identity API snapshot agrees with the local commit object and updated body that gim-home/ModelKitArtifacts#209 is OPEN, Draft, labeled model-scale-by-skill, and changes only bert.json.

The updated remote body exactly matches the frozen source after newline normalization and contains no private path, evidence-root, or handoff-hash leaks. Full REST and cursor-paginated GraphQL enumeration found zero line comments, zero prior formal reviews, zero review threads, and zero open threads; the sole issue comment is the unchanged stale author comment for an older head.

Coverage is full for all four planned CPU task/precision tuples, with no deferred tuples. No merge-blocking findings remain. Leave the PR Draft with model-scale-by-skill.