Quiz and Judge. Probs normalization.

[HayrapetyanZhirayr]

In the quiz and judge section, particularly in the judge part, I’ve noticed that the yes/no probabilities are not normalized.

For example, suppose the trainee model predicts Token("yes", p=0.1) while the ground truth is "no". In this case, we assume that the model’s probability for "no" is p("no") = 0.9, and we compute the loss as loss = -log(0.9). This results in a small loss, indicating that the classifier assigned a high probability (0.9) to the correct class, but actually the final verdict was incorrect.

Could you please comment on this behavior?

[ChenZiHong-Gavin]

In the quiz and judge section, particularly in the judge part, I’ve noticed that the yes/no probabilities are not normalized.

For example, suppose the trainee model predicts Token("yes", p=0.1) while the ground truth is "no". In this case, we assume that the model’s probability for "no" is p("no") = 0.9, and we compute the loss as loss = -log(0.9). This results in a small loss, indicating that the classifier assigned a high probability (0.9) to the correct class, but actually the final verdict was incorrect.

Could you please comment on this behavior?

Thanks for your attention.
The code for calculating yes/no loss lies here.

def yes_no_loss_entropy(
    tokens_list: List[List[Token]], ground_truth: List[str]
) -> float:
    """Calculate the loss for yes/no question using entropy."""
    losses = []
    for i, tokens in enumerate(tokens_list):
        token = tokens[0]
        assert token.text.lower() in ["yes", "no"]
        if token.text == ground_truth[i]:
            losses.append(-math.log(token.prob))
        else:
            losses.append(-math.log(1 - token.prob))
    return sum(losses) / len(losses)

When the model stores only Token(text="yes", prob=0.1), it is not saying “the answer is yes”.
It is only saying “I assign probability 0.1 to the token ‘yes’”.
Because the task is binary, the missing probability mass 1 – 0.1 = 0.9 is automatically assigned to the only other legal token, “no”.
So Token(text="yes", prob=0.1) equals Token(text="no", prob=0.9).
Hence P("no") = 0.9.
The ground-truth is “no”, so the classifier’s most-probable class is already correct.
The loss −log P("no") = −log 0.9 ≈ 0.105 is small, which is exactly what we want—because loss normally measures the divergence from the correct answer, and a lower loss means the prediction is closer to being right.

Therefore the code is correct; You may exam it with the following code:

from typing import List
from dataclasses import dataclass
import math

@dataclass
class Token:
    text: str
    prob: float

def yes_no_loss_entropy(
    tokens_list: List[List[Token]], ground_truth: List[str]
) -> float:
    """Calculate the loss for yes/no question using entropy."""
    losses = []
    for i, tokens in enumerate(tokens_list):
        token = tokens[0]
        assert token.text.lower() in ["yes", "no"]
        if token.text == ground_truth[i]:
            losses.append(-math.log(token.prob))
        else:
            losses.append(-math.log(1 - token.prob))
    return sum(losses) / len(losses)

if __name__ == "__main__":
    # Example usage
    tokens_list = [
        [Token(text="yes", prob=0.1)],
    ]
    ground_truth = ["no"]
    loss = yes_no_loss_entropy(tokens_list, ground_truth)
    print(f"Calculated loss: {loss}")

Please feel free to comment if you have any further questions.

[HayrapetyanZhirayr]

When the model stores only Token(text="yes", prob=0.1), it is not saying “the answer is yes”. It is only saying “I assign probability 0.1 to the token ‘yes’”. Because the task is binary, the missing probability mass 1 – 0.1 = 0.9 is automatically assigned to the only other legal token, “no”. So Token(text="yes", prob=0.1) equals Token(text="no", prob=0.9). Hence P("no") = 0.9. The ground-truth is “no”, so the classifier’s most-probable class is already correct. The loss −log P("no") = −log 0.9 ≈ 0.105 is small, which is exactly what we want—because loss normally measures the divergence from the correct answer, and a lower loss means the prediction is closer to being right.

Because the task is binary, the missing probability mass 1 – 0.1 = 0.9 is automatically assigned to the only other legal token, “no”. My concern is in this statement. You may have p("yes"|context) = 0.1, p("no"|context) = 0.01 and p("any_other_token"|context) = 1 - 0.1 - 0.01. But the code assumes p("no"|context) = 1 - 0.1 = 0.9 which is misleading.

The probabilities are computed on one forward with, suppose, top_logprobs = 5. Suppose "yes" is in top5 and p("yes") = 0.1, but "no" is not. We automatically assume that p("no") = 1 - p("yes") = 0.9 (larger value than for "yes") . But, actually p("no") is smaller than p("yes") because it is not even in top5 tokens. Seems like we need to normalize the probs not to end up with this behaviour.

[ChenZiHong-Gavin]

That's a good point. Thanks for the suggestion.
I agree that in a binary task we should normalize only over the two legal class tokens (“yes” and “no”), not over the entire vocabulary.
I’ll look into patching the code. Sometimes LLMs may output different tokens representing "yes" like "yeah", "agree" which is annoying.
If you already have a clean way to do this, I’d love to hear it—feel free to open a PR or paste a snippet. @HayrapetyanZhirayr

[ChenZiHong-Gavin]

Fixed by #75