Skip to content

Agent Spec Evaluation Computer improvements - #240

Open
soufianeamini wants to merge 1 commit into
mainfrom
evaluation-improvements
Open

Agent Spec Evaluation Computer improvements#240
soufianeamini wants to merge 1 commit into
mainfrom
evaluation-improvements

Conversation

@soufianeamini

Copy link
Copy Markdown
Member
  • Use CapacityLimiter over Semaphore
  • Limit the amount of concurrent tasks that are running at once (the number of tasks spawned would be at most max_concurrency + 1)
  • Producer/worker pattern is not needed anymore as a result

@soufianeamini
soufianeamini requested review from a team and cesarebernardis August 27, 2026 09:07
@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Aug 27, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cesarebernardis The test_unlimited_concurrency test fails if the tasks finish too soon (if I add a bigger delay it succeeds) because it is not required anymore that all tasks start before starting processing. So from testing instead of seeing tasks go up to 200 running tasks concurrently, it ends up hovering around the 60 concurrent tasks range.

Is there a different kind of test you'd want to be implemented or should I just delete it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant