Issues

• Proposal: promptpex-openeval-adapter — export generated tests/results to EvalPort format

  microsoft/promptpex#279 · Filter by author adhabnr-ux opened on Aug 31, 2026

• implement basic python layer

  microsoft/promptpex#177 · Filter by author pelikhan opened on Jun 20, 2025

• Results Overview shows NaN when eval metric score is not a number

  bug
  microsoft/promptpex#174 · Filter by author bzorn opened on Jun 20, 2025
  1

• generating JSON inputs confuses the CSV output

  microsoft/promptpex#165 · Filter by author pelikhan opened on Jun 16, 2025

• Rules model reporting is incorrect

  bug
  microsoft/promptpex#161 · Filter by author bzorn opened on Jun 11, 2025

• testdata is lost with github models

  microsoft/promptpex#158 · Filter by author pelikhan opened on Jun 10, 2025

• add ground truth evaluation

  microsoft/promptpex#157 · Filter by author pelikhan opened on Jun 10, 2025
  

• Metrics are currently saved in the context.

  bug
  microsoft/promptpex#152 · Filter by author bzorn opened on Jun 6, 2025
  

• add documentation page on ground truth

  microsoft/promptpex#149 · Filter by author pelikhan opened on Jun 6, 2025
  

• Disk button icon saves as .txt

  microsoft/promptpex#131 · Filter by author pelikhan opened on May 22, 2025

• support output specification in prompty

  enhancement
  microsoft/promptpex#51 · Filter by author pelikhan opened on Feb 20, 2025
  

• Handle chat prompts that assume a multi-step interaction between user and system

  enhancement
  microsoft/promptpex#25 · Filter by author bzorn opened on Oct 25, 2024

• Feature request: Use the LLM to tag rules that are checkable using code

  enhancement
  microsoft/promptpex#9 · Filter by author bzorn opened on Aug 28, 2024

• Feature request: separate input/output rules into rules related to format/syntax and rules related to content

  enhancement
  microsoft/promptpex#8 · Filter by author bzorn opened on Aug 28, 2024