-
Notifications
You must be signed in to change notification settings - Fork 6
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
[Bug]: theoremqa evaluates model output with bare eval(); the sandbox is escapable, and closing it is a
_fixed-shaped changebugSomething isn't workingSomething isn't workingStatus: Open.#77 In scitix/sieval;[RFC]: Sampled-answer metric family for math and MCQ — avg@k / pass@k / pass^k, consistency, and health signals
RFCRequest for comments on architectural/design changesRequest for comments on architectural/design changesStatus: Open.#74 In scitix/sieval;[Feature]: give JudgementRecord.reference a durable signal for why it is absent
feature requestNew feature or requestNew feature or requestStatus: Open.#71 In scitix/sieval;[RFC]: Observed-serving facts must flow back into dataset packing — the RULER case
RFCRequest for comments on architectural/design changesRequest for comments on architectural/design changesStatus: Open.#58 In scitix/sieval;[RFC]: Capability constraint layer — declare and enforce the silent cross-group constraints
RFCRequest for comments on architectural/design changesRequest for comments on architectural/design changesStatus: Open.#47 In scitix/sieval;[RFC]: Recipe capability layer — split model-type capabilities out of hardware profiles
RFCRequest for comments on architectural/design changesRequest for comments on architectural/design changesStatus: Open.#27 In scitix/sieval;[RFC]: A capability-based Model IR — decouple model access from the OpenAI protocol
RFCRequest for comments on architectural/design changesRequest for comments on architectural/design changesStatus: Open.#25 In scitix/sieval;