Skip to content

Choice answers: choice is sometimes 0.01 below the highest probability (jev-1.13.0) #15

Description

@nevenincs

Summary

For Choice questions, choice is documented as the option with the highest probability (API reference and primitives/choice). With jev-1.13.0 we see responses where choice is exactly 0.01 below another option's probability. It happens only when two options are close to a tie, and looks like the probabilities are rounded to two decimals and renormalised to sum to 1 after the winner is chosen, which can flip a near-tie by one quantum.

Evidence

  • 35 responses on 2026-09-23 (13:00 to 21:35 UTC), all jev-1.13.0, from a production integration that asks 28 questions per request.
  • In every case the gap is exactly 0.01 and the answer is low-confidence (0.09 to 0.42). The probabilities sum to 1.00 in 34 cases and 0.99 in one. The competing options sit between 0.25 and 0.50.
  • Questions affected: update_kind (18), event_kind (5), development_relationship (4), administration_kind (2), opportunity_kind (2), message_format (1), timing (1), event_location (1), language_readable (1). They are all multi-option Choices (4 to 11 options).
Request id Question choice (probability) Highest option (probability) Confidence
req_01a0ce5a0bcb71c69bd2daa0e6f8ca8a event_kind exhibition 0.49 mixed 0.50 0.42
req_01a0ce61c7357e269f56b77051c0c56a message_format event_invitation 0.45 newsletter 0.46 0.36
req_01a0ce673e5f718e9632d4e943c5baf1 update_kind announcement 0.44 reading_recommendations 0.45 0.37
req_01a0ce9858947beaab7d90a6b8fa2d9f event_kind learning 0.48 other_unclear 0.49 0.41
req_01a0cea2857872c1a9b1a754f6b67f33 development_relationship not_technical 0.27 unclear 0.28 0.15
req_01a0cea93ba77197b56043edf842380e update_kind informational_interaction 0.45 routine 0.46 0.38
req_01a0cea9be9678d3864a6833566fe565 update_kind not_an_update 0.48 routine 0.49 0.41
req_01a0ceaffa40779b9c4ffc26c66bc01f timing dated 0.49 unspecified 0.50 0.37
req_01a0ceb8e81f7a228ab45340430bdd1e update_kind not_an_update 0.48 routine 0.49 0.42
req_01a0cebfa74f75b286d0af13e17e4518 update_kind directed_request 0.39 not_an_update 0.40 0.31
req_01a0ceca60dd7d6a9f04bac3da5736e8 administration_kind not_administration 0.44 general_update 0.45 0.39
req_01a0cee582607285bad8849bae98087c update_kind not_an_update 0.48 routine 0.49 0.42
req_01a0cee844c27869a4e81efce53a08ce event_kind not_an_event 0.32 mixed 0.33 0.23
req_01a0ceea374e754b91c01e61abe6be7e update_kind announcement 0.47 not_an_update 0.48 0.41
req_01a0cf095ff27ff8862adf46017fd987 update_kind not_an_update 0.48 routine 0.49 0.42
req_01a0cf0e30497546b7862a582e7e4bb7 opportunity_kind learning 0.37 no_opportunity 0.38 0.27
req_01a0cf18f9e5707bb65c9dde1015bbce development_relationship release 0.43 routine 0.44 0.34
req_01a0cf2614ae7d82ad5bd626a62ee76f update_kind not_an_update 0.39 relevant_change 0.40 0.32
req_01a0cf29820776f1b7591801b321c25f development_relationship unclear 0.28 release 0.29 0.17
req_01a0cf407421740cb579ef1a67c88797 development_relationship routine 0.48 not_technical 0.49 0.40
req_01a0cf4a5d687202a535e1058ca055a2 update_kind not_an_update 0.48 routine 0.49 0.42
req_01a0cf51f7d5710082cdc4cf2970b574 opportunity_kind attendance 0.49 no_opportunity 0.50 0.41
req_01a0cf6d968b70eaae79609664c48315 administration_kind account_service 0.34 general_update 0.35 0.28
req_01a0cf7094e67d2184780c34e9a6c9c5 update_kind reading_recommendations 0.26 announcement 0.27 0.18
req_01a0cf7a77a67089a48f6db071a30d5c update_kind not_an_update 0.47 routine 0.48 0.40
req_01a0cf86daa678779de6971fd1e75457 update_kind not_an_update 0.48 routine 0.49 0.42
req_01a0cf9486af7fc5be1e3c3ec295a25c event_kind not_an_event 0.48 other_unclear 0.49 0.41
req_01a0cf9a16667e05893bf90c8f7febda update_kind not_an_update 0.48 routine 0.49 0.42
req_01a0cfad8be37cb896500bc482ecc7c1 update_kind announcement 0.33 not_an_update 0.34 0.25
req_01a0cfb7ddfa7d02912a1b1add8e112a update_kind not_an_update 0.46 reading_recommendations 0.47 0.40
req_01a0cfc0d65d7092aa1db756298d918a event_location budapest 0.42 not_an_event 0.43 0.34
req_01a0d01572347c9984bb5017a8112060 language_readable readable 0.39 unreadable 0.40 0.09
req_01a0d02436a275788d6492d2eb34c05a event_kind not_an_event 0.41 talk 0.42 0.32
req_01a0d02c948477b3af65b8cc02d41143 update_kind informational_interaction 0.24 not_an_update 0.25 0.15
req_01a0d031716473e58c07731be0bccd8c update_kind not_an_update 0.48 routine 0.49 0.42

Expected

choice equals the option with the highest probability, as documented. Alternatively, if ties within rounding are intended, the docs could say that choice may differ from the rounded argmax by one rounding step.

Impact

Clients that trust the documented contract have to discard these answers. We now drop only the affected answer, so the impact is small, but it's a silent inconsistency that's easy to miss.

Environment

typesafe-sdk 0.7.1 (Python), model jev-1.13.0, /v1/systemone. We can share more detail about specific requests privately if that helps.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions