[NeurIPS'2026] Position Bias in Mamba and Hybrid Language Models | Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering
-
Updated
Aug 22, 2026 - Python
#
position-bias
Here are 17 public repositories matching this topic...
Seven decision models, measured as the candidate list grows, the option order changes, and the wrong answers stop being obvious.
calibration typesafe reproducibility robustness model-comparison intent-classification laya zero-shot-classification llm openrouter decision-models position-bias jev
-
Updated
Sep 25, 2026 - Python
ACL 2026 | CapCal: content-agnostic probability calibration for de-biasing listwise rerankers.
-
Updated
Jun 10, 2026 - Python
Do LLM judges know when they're wrong? Three open-weight judges (Qwen2.5-7B, kev-8b, auto-j-13b) graded against MT-Bench human votes: calibration, position and padding attacks, and what auto-accepting confident verdicts would cost. Interactive site included.
python machine-learning statistics evaluation data-visualization calibration uncertainty-quantification bayesian-statistics interactive-visualizations numpyro selective-prediction llm vllm llm-eval qwen llm-evaluation llm-as-judge llm-as-a-judge mt-bench position-bias
-
Updated
Sep 25, 2026 - Python
Learning-to-rank lab: listwise models, position bias, calibration, and NDCG evaluation
-
Updated
Sep 21, 2026 - Python
Option-reversal control for paired binary vision–language benchmarks: estimates a model's answer-slot bias and perceptual accuracy from one extra inference pass per question, attributes each failure to slot or perception, and reports scores against the correct chance floors. Ongoing project; code only.
pytorch selection-bias huggingface evaluation-methodology multimodal-llm vision-language-models benchmark-evaluation position-bias mmvp
-
Updated
Sep 19, 2026
position bias in LLM judges is worse than people think: ask a small instruct model to choose 1 of 2 items. 81% of the answer will be based on which slot the item was in, not which it was. swap the order and the answer flips 78% of the time. an inconclusive cognitive dissonance experiment
-
Updated
Jul 29, 2026 - Python
Position-bias-aware ranking: estimating and correcting position bias to optimize Earnings Per Visitor (EPV) · simulation study · IPW & propensity modeling · Python
python data-science machine-learning simulation ranking ab-testing recommender-system causal-inference propensity-score off-policy-evaluation inverse-propensity-weighting position-bias counterfactual-evaluation
-
Updated
Jun 6, 2026 - Jupyter Notebook
Blind side-by-side human eval of two model responses on a weighted rubric — randomized panes to kill position bias, bootstrap CIs on the margin, inter-rater reliability. An eval you can't audit is a vote, not a measurement.
product-management human-in-the-loop ai-quality llm-evaluation single-file-app inter-rater-reliability position-bias rubric-scoring
-
Updated
Sep 18, 2026 - JavaScript
SSIT: a label-free, gold-free test for whether LLM-judge position x verbosity bias corrections actually compose. No human labels, no model of the judge. Code, 7-judge/6-family pilot data, and pre-registered protocol for the NeurIPS 2026 JUDGe workshop paper.
bias-correction difference-in-differences ssit llm-evaluation llm-as-judge neurips-2026 position-bias verbosity-bias
-
Updated
Sep 16, 2026 - Python
Measure position/verbosity/assertiveness bias in an LLM-as-judge (Claude) by judging pairs in both orders. Finding: no position bias, but 75% verbosity bias and 100% assertiveness bias — judge scores gameable by length + tone.
-
Updated
Jul 25, 2026 - Python
Two kinds of saturation: why LLM-judge order bias is hard to measure — essay, Lean proofs (0 sorry), and a reproducible dispersion measurement.
reproducible-research evaluation formal-verification quantum-cognition lean4 order-effects llm-as-a-judge position-bias
-
Updated
Aug 23, 2026 - Python
Official reproduction code for 'Efficient LLM Preference Classification Through Position Bias Mitigation and Architectural Symmetry' (Applied Intelligence APIN-D-26-05359)
-
Updated
Sep 14, 2026 - Python
Counterfactual learning-to-rank for marketplace search logs in PySpark: position-bias estimation, IPS-weighted training, NDCG evaluation against known ground truth
machine-learning spark pyspark learning-to-rank recommender-systems counterfactual-learning position-bias
-
Updated
Sep 6, 2026 - Python
Open-source implementation of the typed-decision pattern popularised by TypeSafe's Jev: read a decision out of a small language model's logits, in the browser. Library, benchmarks and paper. Not affiliated with TypeSafe.
python lora webgpu fine-tuning structured-output on-device-ai onnx-runtime constrained-decoding llm-inference llm-evaluation small-language-models transformers-js qwen3 browser-ml position-bias jev typed-decisions logit-readout
-
Updated
Sep 21, 2026 - Python
Monthly bias audits of LLM judges
-
Updated
Sep 6, 2026 - Python
Ranking evaluation with error bars: NDCG, MRR and MAP with confidence intervals, plus position-bias correction for click logs. No dependencies.
search information-retrieval evaluation ranking learning-to-rank recommender-systems ndcg position-bias
-
Updated
Sep 6, 2026 - Python
Add this topic to your repo
To associate your repository with the position-bias topic, visit your repo's landing page and select "manage topics."