sync_inference_weights corrupts the sampler for dense Qwen3 models with tied embeddings

[qywu]

Minimal repro (dense Qwen3-4B-Instruct-2507, tie_word_embeddings: true):

1. Launch the trainer in full-weight mode on the exact dense-Qwen3 contract (ce_mode: bi_fused, fa4, merge_qkv: false) and xorl-sglang with --rl-on-policy-target xorl --enable-fp32-lm-head on the same model.
2. Fresh sampler generates perfectly (<guess>CRANE</guess> for the Wordle prompt).
3. add_inference_endpoint + sync_weights_to_inference(sync_method='nccl_broadcast') from the untrained trainer — i.e. byte-identical weights. Sync reports success: Synced 399 params to 1 endpoint(s), 8,822,848,512 bytes.
4. The same prompt now yields deterministic garbage ('arn讹tereg游客ashingtonoins Prompt trứng…' — identical string across repeated syncs/restarts, so the sync deterministically writes wrong bytes).

Likely suspect: tied embeddings. Qwen3-4B has tie_word_embeddings: true (Qwen3-8B does not); a scrambled lm_head/embedding readout produces exactly this symptom (coherent-frequency random tokens from step one). Possibly interacting with the exact-contract trainer's wrapped/Class-B parameter representations at extraction time.

Impact: blocks the endpoint-driven Wordle zero-K3 recipe (xorl-client examples/wordle) on tied-embedding dense models via full-weight sync. Found while reproducing the train-inference-mismatch write-up's Wordle experiment (see #84 thread for the K₃ measurements that motivated the full-weight route).

Env: 2×H100, torch 2.11, branch qywu/value-head-critic (sync path identical to main).

[qywu]

Scope update — not tied-embeddings-specific. The same repro corrupts a Qwen3-8B sampler (tie_word_embeddings: false): fresh sampler generates perfect <guess>CRANE</guess>; after one nccl_broadcast sync from an untrained full-weight trainer, the same seeded prompt returns deterministic garbage. So the corruption is general to the dense-Qwen3 lane, and since _qwen3_dense_exact_contract engages unconditionally for server_training and model_type == 'qwen3' (src/xorl/models/auto.py:878, no opt-out), full-weight sync is currently broken for every dense Qwen3 server-training config on this tree.

Likely locus: the trainer-side parameter extraction under the exact contract (Class-B RoPE / batch-invariant wrapped trunk / rmsnorm_mode: sglang_fused representations) shipping transformed bytes that the sampler loads as plain HF weights. Two clean isolation levers for whoever picks this up: (1) diff one synced tensor against the checkpoint on the sampler side; (2) the corruption is deterministic — identical garbage across repeated sync+probe cycles.

This blocks the xorl-client examples/wordle zero-K3 recipe end-to-end for dense Qwen3 (both 4B and 8B verified). The rest of the recipe pipeline is confirmed working up to this point on this tree: session creation, endpoint registration/dereg, grouped rollouts (after four small client/example compat patches worth upstreaming: transformers-version-safe apply_chat_template tokenization, enable_thinking=False for hybrid Qwen3, nccl_broadcast in the config schema, and omitting the hardcoded lora_config in full-weight create_model).

[qywu]

Root cause found and fixed in #88.

Not tied embeddings after all — the corruption reproduces on untied Qwen3-8B too. The actual bug: build_training_model(merge_qkv=False) (non-LoRA) unfused attention via the per-layer layer.self_attn.unfuse_for_tp() loop, which never sets model._unfused_for_tp. The checkpoint handler keys its qkv/gate_up merge skipping on that flag, so the loader kept emitting merged qkv_proj keys that the freshly created (empty, meta-init) unfused q/k/v modules could never receive — every attention weight silently loaded as garbage. Weight sync then faithfully shipped those bytes to the sampler; the sync path itself was never at fault.

Verification:

• Sender-side byte diff of _extract_params_for_sync output vs the HF checkpoint: 108/399 corrupted tensors (q/k/v of all 36 layers, fracbad ≈ 0.74) before → 0/399 after.
• Live nccl_broadcast sync from an untrained trainer now leaves the SGLang sampler generating coherently.
• The xorl-client examples/wordle step-1 correctness gate now produces real readings: K₃ ≈ 7.7e-3, mean ratio error ≈ 0.127 on this tree (torch 2.11 / fa4 / bi_fused CE). That's kernel-level train↔inference residual — better than the mismatch write-up's unaligned baseline (~0.017 K₃), but the literal-zero zero_k3 gate needs the write-up's bitwise-aligned kernel trees, which is a separate alignment work item, not weight corruption.

Regression test tests/models/test_merge_qkv_unfuse_load.py covers the handler-flag contract on CPU.