Skip to content

Add client-side timeout and safe comparison for Python challenge runs#18

Merged
ranuru merged 3 commits into
developmentfrom
copilot/fix-infinite-eval-loop
May 12, 2026
Merged

Add client-side timeout and safe comparison for Python challenge runs#18
ranuru merged 3 commits into
developmentfrom
copilot/fix-infinite-eval-loop

Conversation

Copilot AI commented May 12, 2026

Copy link
Copy Markdown
Contributor

User-submitted Python solutions were able to hang indefinitely despite the intended 5s limit, and incorrect outputs (e.g., None) were not always handled safely. The system should always return the actual output or a timeout error within 5 seconds.

  • Timeout enforcement (Python runs): introduce a dedicated Python runner with an AbortController-backed 5s client timeout and shared timeout constants to keep behavior consistent with JS runs.
  • Result validation: guard schema-matching comparisons to fail cleanly when outputs are not arrays.

Example:

const RUN_TIMEOUT_MS = 5000
const RUN_TIMEOUT_MESSAGE = 'Timeout (5s)'

async function runPython(harnessTemplate, userCode, input) {
  const controller = new AbortController()
  const timeout = setTimeout(() => controller.abort(), RUN_TIMEOUT_MS)
  // ...
}

Copilot AI linked an issue May 12, 2026 that may be closed by this pull request
Copilot AI and others added 2 commits May 12, 2026 13:52
Agent-Logs-Url: https://github.com/ranuru/schema-lab/sessions/0610e907-62a8-4f6e-af9d-51acd386714f

Co-authored-by: ranuru <126164337+ranuru@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix infinite eval loop on incorrect solutions Add client-side timeout and safe comparison for Python challenge runs May 12, 2026
Copilot finished work on behalf of ranuru May 12, 2026 13:56
Copilot AI requested a review from ranuru May 12, 2026 13:56
@ranuru ranuru marked this pull request as ready for review May 12, 2026 14:54
@ranuru ranuru marked this pull request as draft May 12, 2026 14:56
@ranuru ranuru marked this pull request as ready for review May 12, 2026 14:57
@ranuru ranuru merged commit 46c19e2 into development May 12, 2026
1 check passed
@ranuru ranuru deleted the copilot/fix-infinite-eval-loop branch May 12, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

infinite eval loop when entering incorrect solutions

2 participants