Skip to content

fix: prevent fleet snapshot argument overflow - #2812

Open
unhexquadium wants to merge 1 commit into
kunchenguid:mainfrom
unhexquadium:fm/fm-fleet-view-jq-arglist
Open

fix: prevent fleet snapshot argument overflow#2812
unhexquadium wants to merge 1 commit into
kunchenguid:mainfrom
unhexquadium:fm/fm-fleet-view-jq-arglist

Conversation

@unhexquadium

Copy link
Copy Markdown

Intent

Fix fm-fleet-view/fm-fleet-snapshot failing with jq 'Argument list too long' at bin/fm-fleet-snapshot.sh:635 once the fleet reaches 17+ task metas, reproducing the current 29-meta failure first. Move all fleet-sized JSON payloads off jq argv and onto stdin or equivalent streamed/file transport while keeping normal-case output byte-identical. Sweep bin/fm-fleet-snapshot.sh and bin/fm-fleet-view.sh for any other jq or xargs call whose argv grows with fleet size and fix those in the same pass. Add a tests/fm-fleet-snapshot-view.test.sh regression using a large synthetic fleet sufficient to trip the old limit. Do not merge the PR. Push the feature branch to the fork over SSH at git@github.com:unhexquadium/firstmate.git, open the PR against upstream kunchenguid/firstmate, and preserve this exact line under a ## Pipeline section in the PR body: Updates from git push no-mistakes. Use a concise top-level summary, a dedicated Risk assessment section with a green/yellow/red status-color emoji, and collapsible details for per-item breakdowns, background, validation notes, and caveats; refer to the captain as she/her. Drive all no-mistakes gates through PR and CI, and when the fork-PR workflow is awaiting upstream maintainer approval after the pipeline finishes, report the PR as delivered with CI awaiting that approval.

What Changed

  • Stream task, inventory, secondmate, and final snapshot JSON into jq through stdin instead of passing fleet-sized payloads through --argjson arguments.
  • Add a 24-task synthetic fleet regression with large decision payloads that verifies both snapshot generation and fleet view rendering.

Risk Assessment

✅ Low: Captain, the change is a focused transport refactor that moves every fleet-sized JSON value in both snapshot modes off jq argv while preserving the existing JSON structure and rendering path.

Testing

The focused behavior suite passed, the base commit reproduced jq: Argument list too long at line 635 with 29 metas, and the target preserved all 29 tasks through snapshot JSON, secondmate-summary, and rendered-view paths while remaining byte-identical for a normal two-task fleet; the CLI evidence was captured and the worktree remained clean.

Evidence: 29-meta base-versus-target CLI transcript

Source: 29-meta base-versus-target CLI transcript

$ base fm-fleet-snapshot.sh --json with 29 task metas and 40000-byte decision summaries
exit=1
/tmp/fm-large-fleet-e2e.SrGnsW/base-bin/fm-fleet-snapshot.sh: line 635: /usr/bin/jq: Argument list too long
fm-fleet-snapshot: main inventory summary failed

$ target fm-fleet-snapshot.sh --json with the same 29-meta fleet
{
  "schema": "fm-fleet-snapshot.v1",
  "task_count": 29,
  "max_decision_summary_bytes": 40000,
  "main_inventory_valid": true
}

$ target fm-fleet-snapshot.sh --secondmate-home-summary with the same 29-meta fleet
{
  "schema": "fm-secondmate-home-summary.v1",
  "endpoint_count": 29,
  "decision_count": 29,
  "shown_endpoints": 20,
  "omitted": [
    {
      "surface": "decisions_open",
      "count": 9
    },
    {
      "surface": "endpoints",
      "count": 9
    }
  ]
}

$ target fm-fleet-view.sh with the same 29-meta fleet (representative rendered rows)
# Fleet View

Schema: fm-fleet-snapshot.v1
Home: /tmp/fm-large-fleet-e2e.SrGnsW/home

## Under Way
| ID | Current | Kind | Repo/Project | Backend | Endpoint | Artifact | Path | Watch / return channel |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| large-task-01 | unknown / none | ship | alpha | tmux | present | - | - | bin/fm-peek.sh fm-large-task-01 |
| large-task-02 | unknown / none | ship | alpha | tmux | present | - | - | bin/fm-peek.sh fm-large-task-02 |
| large-task-03 | unknown / none | ship | alpha | tmux | present | - | - | bin/fm-peek.sh fm-large-task-03 |
| large-task-29 | unknown / none | ship | alpha | tmux | present | - | - | bin/fm-peek.sh fm-large-task-29 |

$ byte-for-byte normal-case comparison: base vs target with two ordinary task metas
identical=yes
sha256=23d0e11154effa398116f6989c202a4c0b1f0df694ad341276afbb54f61916dd

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • tests/fm-fleet-snapshot-view.test.sh
  • /home/kalvira/.no-mistakes/evidence/01M0NSSEMQB2TYB1GZFWXJ9RF8/large-fleet-e2e.sh "$PWD" 1231b6ae7fd4c5ff7e94d2f5ca2159536c4c41cb
  • rg -n "OUTPUT_MODE|secondmate-home-summary|jq |xargs" bin/fm-fleet-snapshot.sh bin/fm-fleet-view.sh
  • git status --short
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code failure identified.

The streamed inputs remain valid, positionally stable JSON documents, the refactored field mappings preserve their prior semantics, and the regression exercises the argument-overflow path through both snapshot and view generation.

Reviews (1): Last reviewed commit: "Fix fleet snapshot jq argument scaling" | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant