Skip to content

feat: add --with-ai options for generate-arazzo command - #3052

Open
DmitryAnansky wants to merge 11 commits into
mainfrom
feat/generate-arazzo-with-ai
Open

feat: add --with-ai options for generate-arazzo command#3052
DmitryAnansky wants to merge 11 commits into
mainfrom
feat/generate-arazzo-with-ai

Conversation

@DmitryAnansky

@DmitryAnansky DmitryAnansky commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

What/Why/How?

  • Added --with-ai and --ai-provider options for generate-arazzo command .
  • Fixed respect and the x-security-scheme-required-values rule rejecting x-security HTTP schemes written with non-lowercase casing (such as Basic, Bearer, or Digest) — RFC 7235 scheme names are case-insensitive.

Notes:

  • Use x-security when possible.
  • Provide hint with placeholders how to execute generated workflow.
  • Complex security flows are not supported due to lack of initial information to reconstruct it.
  • Two-pass design generation for big descriptions like GITHUB API.

Reference

https://github.com/Redocly/redocly/issues/25890

Testing

Screenshots (optional)

Check yourself

  • This PR follows the contributing guide
  • All new/updated code is covered by tests
  • Core code changed? - Tested with other Redocly products (internal contributions only)
  • New package installed? - Tested in different environments (browser/node)
  • Documentation update has been considered

Security

  • The security impact of the change has been considered
  • Code follows company security practices and guidelines

Note

Medium Risk
Optional --with-ai sends resolved OpenAPI to a local AI CLI (data-exposure risk if specs contain secrets), and changes to generated security/x-security wiring can alter how new Arazzo files run under respect.

Overview
generate-arazzo gains optional --with-ai (plus --ai-provider, --ai-model, --max-workflows) to turn the mechanical one-operation-per-workflow skeleton into multi-step scenarios via locally installed claude, codex, or cursor CLIs. AI output is constrained (baseline header preserved, operation refs checked, spec lint) and the command falls back to the baseline on failure; large OpenAPI docs use a two-pass flow (scenario selection, then per-scenario design). Successful AI runs prepend a verification comment to the YAML.

After any generation, the CLI now prints a copy-paste respect command with --input placeholders for each workflow input (including resolved component $refs).

Respect / linting now treat HTTP auth scheme names as case-insensitive (Basic, Bearer, etc.) in x-security-scheme-required-values and runtime validation.

The Arazzo auto-generator bumps to arazzo: 1.1.0, returns a structured TestDescription from respect-core generate(), and reworks security: step-level x-security with proper Basic (username/password) inputs, OAuth2/OIDC tokens, and fewer unsupported schemes; workflow-level Authorization hacks are removed. Shared AI helpers (stripCodeFences, lintDocumentSource, providers) move under utils/ai for reuse with generate-spec.

Reviewed by Cursor Bugbot for commit bce13b9. Bugbot is set up for automated code reviews on this repo. Configure here.

@changeset-bot

changeset-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: bce13b9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@redocly/cli Minor
@redocly/respect-core Minor
@redocly/openapi-core Minor
@redocly/client-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@DmitryAnansky DmitryAnansky self-assigned this Aug 25, 2026
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Performance Benchmark (Lower is Faster)

CLI Version Bundle Lint Check Config
cli-latest ▓ 1.00x (Fastest) ▓ 1.00x ± 0 ▓ 1.00x ± 0.01
cli-next ▓ 1.01x ± 0.02 ▓ 1.00x (Fastest) ▓ 1.00x (Fastest)

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 78.75% (🎯 77%) 14217 / 18052
🔵 Statements 78.51% (🎯 77%) 15310 / 19499
🔵 Functions 82.38% (🎯 81%) 2914 / 3537
🔵 Branches 71.72% (🎯 71%) 10538 / 14692
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/cli/src/types.ts 100% 100% 100% 100%
packages/cli/src/commands/generate-arazzo/index.ts 93.33% 89.28% 80% 93.33% 47, 90-91
packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts 89.4% 74.25% 90.62% 90.08% 64, 68, 74, 78, 84, 147, 187, 197, 207, 210, 241-245, 248, 251-253, 266, 269, 303, 315, 317, 328, 421, 428, 540, 555
packages/cli/src/commands/generate-arazzo/ai/prompt.ts 96.29% 94.73% 100% 96.15% 185-189
packages/cli/src/commands/generate-spec/index.ts 0% 0% 0% 0% 31-113
packages/cli/src/commands/generate-spec/ai/refine.ts 97.1% 86.11% 100% 97.05% 106, 137, 141, 240
packages/cli/src/utils/ai/output.ts 100% 100% 100% 100%
packages/core/src/rules/respect/x-security-scheme-required-values.ts 90.9% 93.75% 100% 90.9% 40-43
packages/respect-core/src/generate.ts 0% 100% 0% 0% 17
packages/respect-core/src/modules/arazzo-description-generator/generate-arazzo-description.ts 100% 85.71% 100% 100%
packages/respect-core/src/modules/arazzo-description-generator/generate-inputs-arazzo-components.ts 100% 100% 100% 100%
packages/respect-core/src/modules/arazzo-description-generator/generate-step-security.ts 94.73% 90.9% 100% 94.73% 32
packages/respect-core/src/modules/arazzo-description-generator/generate-workflows-from-description.ts 100% 76.66% 100% 100%
packages/respect-core/src/modules/context-parser/get-security-parameters.ts 100% 100% 100% 100%
packages/respect-core/src/modules/flow-runner/validate-x-security-parameters.ts 100% 100% 100% 100%
packages/respect-core/src/utils/api-fetcher.ts 55.1% 45.37% 73.33% 54.79% 88-94, 124, 151-167, 178-179, 189-192, 197-200, 205, 208-229, 242-255, 263, 310-316, 323-401, 409, 417, 438, 453, 460, 463, 467
Generated in workflow #11542 for commit bce13b9 by the Vitest Coverage Report Action

@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai and --ai-provider options for generate-arazzo command [POC] feat: add the experimental --with-ai options for generate-arazzo command [POC] Aug 25, 2026
@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai options for generate-arazzo command [POC] feat: add the experimental --with-ai options for generate-arazzo command (POC) Aug 25, 2026
Comment thread .changeset/generate-arazzo-with-ai.md Outdated
@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai options for generate-arazzo command (POC) feat: add the experimental --with-ai options for generate-arazzo command Aug 26, 2026
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch 3 times, most recently from c3b2dec to 37a0234 Compare August 26, 2026 14:05
@DmitryAnansky
DmitryAnansky marked this pull request as ready for review August 27, 2026 10:01
@DmitryAnansky
DmitryAnansky requested review from a team as code owners August 27, 2026 10:01
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts Outdated
@DmitryAnansky DmitryAnansky changed the title feat: add the experimental --with-ai options for generate-arazzo command feat: add --with-ai options for generate-arazzo command Aug 27, 2026
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch from 2709a88 to c39b251 Compare August 27, 2026 14:30
@DmitryAnansky
DmitryAnansky force-pushed the feat/generate-arazzo-with-ai branch from 50c3833 to 24e1f7a Compare August 27, 2026 15:38
Comment thread .changeset/generate-arazzo-respect-hint.md Outdated
Comment thread .changeset/generate-arazzo-with-ai.md Outdated
Comment thread .changeset/respect-scheme-case.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
Comment thread docs/@v2/commands/generate-arazzo.md Outdated
@DmitryAnansky
DmitryAnansky requested a review from JLekawa August 28, 2026 08:07

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a1289ec. Configure here.

Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts
Comment thread packages/cli/src/commands/generate-arazzo/ai/generate-workflows.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants