Skip to content

test: add integration test for allow_queue on job execution create - #227

Merged
drappier-charles merged 1 commit into
mainfrom
cdrappier/devin/allow-queue-integration-test
Aug 21, 2026
Merged

test: add integration test for allow_queue on job execution create#227
drappier-charles merged 1 commit into
mainfrom
cdrappier/devin/allow-queue-integration-test

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #226: adds the integration test Devin Review flagged as missing (AGENTS.md requires every feature to ship with one).

test_run_job_with_allow_queue_false in tests/integration/core/jobs/test_jobs.py runs job.arun(tasks=[...], allow_queue=False) following the existing memory/env override test patterns. Since allow_queue=False legitimately returns 429 when the workspace has no capacity at that moment, a 429 is treated as the expected immediate-rejection behavior (skip) rather than a failure; otherwise the created execution is verified via aget_execution.

Link to Devin session: https://app.devin.ai/sessions/829498efd05742f79300261953889d6d
Requested by: @drappier-charles


Note

Low Risk
Test-only change; no production or security-sensitive code is modified.

Overview
Adds an integration test that runs a job with allow_queue=False via job.arun.

If capacity is available, it asserts the execution is created; if not, a 429 is treated as expected immediate rejection (skip) rather than a failure. Matches the existing arun override test pattern.

Reviewed by Cursor Bugbot for commit aa50418. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@drappier-charles
drappier-charles marked this pull request as ready for review August 21, 2026 20:44
@drappier-charles
drappier-charles merged commit c7f7ccf into main Aug 21, 2026
18 of 19 checks passed
@drappier-charles
drappier-charles deleted the cdrappier/devin/allow-queue-integration-test branch August 21, 2026 20:44

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit aa50418. Configure here.

# Verify execution was created
execution = await job.aget_execution(execution_id)
assert execution is not None
assert execution.status is not None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

False pass on queued executions

Medium Severity

The success path only checks that an execution exists and has some status. Per the allow_queue=False contract, a capacity miss must 429 rather than queue, but a queued execution still passes here while a correct 429 is only skipped—so ignoring allow_queue under capacity pressure is not caught.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit aa50418. Configure here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but this can't be asserted deterministically from an integration test: the "capacity unavailable" branch only exists under real capacity pressure, which the test can't force in the shared workspace. When capacity is available, the outcome is identical with or without allow_queue=False (an execution is created), and immediately after create the stored status is legitimately queued/pending before admission, so asserting status != queued on the success path would flake rather than catch an ignored flag. The contract itself (synchronous check, immediate 429, no record) is covered by unit tests on the controlplane side (blaxel-ai/controlplane#5229, register_test.go / router tests). This test's purpose is the AGENTS.md requirement to exercise the new SDK parameter end to end against the real API — verifying the field serializes and the API accepts it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant