Skip to content

🪟 feat: Preserve Command Output Heads and Tails - #275

Merged
danny-avila merged 3 commits into
mainfrom
lia/command-output
Oct 2, 2026
Merged

danny-avila merged 3 commits into
mainfrom
lia/command-output

Conversation

@lia-by-librechat

Copy link
Copy Markdown
Contributor

Summary

Attached native commands currently keep only the first output bytes that reach a shared stdout/stderr budget. Long test runs lose their final summaries, and noisy stdout can consume the entire allowance before an error reaches stderr.

Keep a bounded prefix and rolling suffix for each stream, then fit both into the existing combined maxOutputBytes limit. Inline [... N bytes omitted ...] markers report dropped raw bytes. Stdout and stderr remain separate; quiet streams donate unused space, while noisy streams split the budget with the odd byte reserved for stderr. Sandbox violation annotations enter stderr before final rendering.

This is B1 only. LibreChat A1/A2 remain in #16539 and #16540. No parent creation, read/search changes, edits, model-family selection, or patch execution is included. This does not depend on worker #271.

Mechanism

child stdout / stderr
  → per-stream copied prefix + fixed-capacity suffix ring
  → append sandbox diagnostics to stderr
  → balance stream budgets and render UTF-8-safe windows + omission markers
  → unchanged execute_command result and existing validators
  • Retained raw output is at most twice maxOutputBytes, independent of total output and chunk count. No references to oversized child buffers or per-chunk buffer arrays survive capture.
  • UTF-8 boundaries, malformed-byte replacement expansion, and omission markers fit the combined response byte budget.
  • Tiny stream allowances that cannot fit an omission marker still return the existing truncated flag.
  • Small output remains unchanged. Truncation never stops execution. Timeout/signal metadata, cancellation errors, mutation certainty, and cleanup remain unchanged.
  • No new protocol, capability, configuration, or response keys. Existing Code API and LibreChat readers accept the same result shape; no reader-first rollout is required.

Verification

Focused checks and exact-head review results are recorded in the head handoff comment.

  • Collector tests cover chunk-independent suffix retention, exact omission counts, stream fairness, quiet-stream donation, tiny limits, UTF-8 and invalid bytes, exact-fit output, and copying rather than retaining source chunks.
  • Native command regressions use real Bash for both-stream summaries, sandbox annotations, and timeout settlement, and validate the legacy result shape.
  • Local collector-boundary probes run real Bash directly through the production spawned-process collector. Summary and timeout probes fail on the previous collector and pass with B1; cancellation semantics pass on both.
  • The local public native-sandbox.test.ts suite is blocked before command execution by the existing private-storage ancestor guard because this worker sandbox's / belongs to uid 65534. Unchanged main reproduces that failure. The guard was not weakened. Collector-boundary probes do not certify SRT admission or confinement.

Not run locally: the complete packages/code suite, live SRT confinement tests, service/API builds or Bun suites. Only packages/code source changes. No deployment or benchmark improvement is claimed.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

B1 head handoff

Pushed head: f11f51d036d04199f9036531003822f2b7367b6e
Base and merge-base: 3189cfd8ee7f81cf90671f9b91b1d52ae9d3693d

Exact-head local check Result
packages/code: tsc --noEmit Passed
packages/code: real npm run build Passed
output.test, native-process.test, protocol.test 55 passed, 0 failed
Real Bash through the production spawned-process collector 3 passed: both-stream summaries/annotations/legacy validator, timeout settlement, typed cancellation
New collector/test Prettier check and git diff --check Passed

The summary and timeout probes fail on the previous collector; cancellation remains unchanged. The local public native sandbox suite stops at the existing private-storage ownership guard before command execution, and unchanged main reproduces that root-ownership rejection. CI's three full code-package matrix lanes and Linux native-sandbox lane passed this exact head.

Existing native-sandbox.ts, native-sandbox.test.ts, and the README already fail full-file Prettier on the base revision. Newly added test blocks were formatted and unrelated formatting was preserved. There is no packages/code ESLint/import-sort task. The complete local code-package suite, live local SRT confinement tests, and local service/API builds or Bun suites were not run.

Independent review was requested for this exact head. The child runner failed to complete and returned no review verdict or finding ledger. This is not a clean review; severity counts and dispositions are unavailable. No GitHub inline review findings were present at the handoff check.

A1/A2 PRs are untouched. No deployment, merge, or benchmark improvement is claimed.

@LibreChat-AI LibreChat-AI deleted a comment from lia-by-librechat Bot Sep 30, 2026
@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review

Please review B1 at exact pushed head f11f51d036d04199f9036531003822f2b7367b6e against merge-base 3189cfd8ee7f81cf90671f9b91b1d52ae9d3693d.

Scope: bounded head-and-tail native command output, separate stdout/stderr, stderr reservation, inline omission counts, UTF-8/combined-budget enforcement, and unchanged timeout/cancellation/result contracts. A1/A2 and filesystem/search/patch slices are excluded.

Local focused collector/protocol/subprocess checks and real-Bash collector probes pass. Public native sandbox tests are blocked before execution by the existing private-storage ancestor guard in this worker; unchanged main reproduces the same root-ownership rejection. Full CI and independent exact-head review run separately.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-30T10:39:07.476427Z f11f51d Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f11f51d036

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/code/src/output.ts Outdated
Merge current main while preserving Git metadata denies and tolerant edit support. Retain malformed quiet-stream output beside noisy output within the combined byte budget.
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Updated head: ee97d050cd04f8213b7372ad4754cff913637122
Base/merge-base: 836d001319015072f9493df5d7710fc371403962

Merged current main without rewriting history. Both sides of the README conflict are retained, including upstream Git-metadata security guidance. Fixed P2 GH-4143658407 with rendered-byte stream allocation and malformed quiet-stream regressions.

Focused checks, CI, and an independent review are running for this exact head. Existing A1/A2 PRs are untouched.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

B1 update

Head: ee97d050cd04f8213b7372ad4754cff913637122
Base/merge-base: 836d001319015072f9493df5d7710fc371403962

Merged current main without rewriting history. Both README sections are retained; upstream Git-metadata protections and tolerant edits are unchanged. GitHub reports the conflict resolved.

Exact-head local check Result
packages/code: tsc --noEmit and npm run build Passed
output.test, native-process.test, protocol.test 59 passed
Real-Bash collector probes 5 passed: summaries, annotations, timeout, cancellation, malformed quiet stdout/stderr
Deterministic arbitrary-byte/chunk invariant sweep 5,000 cases passed
New collector/test formatting and base-to-head whitespace check Passed

Finding ledger: GH-4143658407, P2, quiet malformed output starved by raw-byte allocation, fixed in this head. The new regression fails on the reviewed collector and passes with rendered-byte allocation. Replied to and resolved the thread. No finding was rejected.

The four selected public native-command tests remain blocked locally before execution by the existing private-storage ancestor guard (/ owned by uid 65534). The guard is unchanged. CI's full Node 20/22/24 code-package matrix and Linux native-sandbox lane passed this head. Local collector probes do not certify SRT admission or confinement.

Not run locally: the complete code-package suite, live SRT confinement tests, service/API builds and Bun suites. Full-file Prettier still reports pre-existing formatting in the native sandbox files and README; new blocks were formatted without unrelated rewrites. There is no code-package lint/import-sort task.

A1/A2 PRs are untouched. No merge to main, deployment, or benchmark improvement is claimed.

CI at handoff: 10 checks passed; only Lambda MicroVM Runner Image (arm64) remains pending, with no failures. Independent review of this exact head is still running; no verdict is claimed.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head: a99b89ab7a298575c00374a60968d560d8a73db9
Base/merge-base: 836d001319015072f9493df5d7710fc371403962

Independent review found P2 IR-SUFFIX-1: a partially filled tail dropped a summary whose bytes remained in the head. Fixed in this head. Retained buffers now render as a logical contiguous stream until capture overflows. New boundary/UTF-8 invariants and real-command regressions cover both streams and stream orders; the summary and logical-window tests fail on ee97d05 and pass here.

P2 GH-4143658407 remains fixed in ee97d05. Neither finding was rejected. The subsystem audit followed capture, allocation, annotation, IPC/protocol validation, programmatic output, timeout, cancellation, and cleanup.

Local build/typecheck, 62 focused tests, nine real-Bash collector probes, 5,000 arbitrary-byte/chunk cases, and scoped formatting pass. Public SRT admission remains subject to the unchanged local storage-ownership blocker. CI and a fresh independent review are running for this exact head. A1/A2 are untouched.

@lia-by-librechat

lia-by-librechat Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor Author

B1 verification

Head: a99b89ab7a298575c00374a60968d560d8a73db9
Base/merge-base: 836d001319015072f9493df5d7710fc371403962

Exact-head local check Result
packages/code: tsc --noEmit Passed
packages/code: npm run build Passed
output.test, native-process.test, protocol.test 62 passed
Real-Bash collector probes 9 passed, including both stream orders for the 129-byte summary
Arbitrary-byte/chunk invariant sweep 5,000 cases passed
Full-source UTF-8 retention oracle 3,888 cases passed
New collector/test formatting and base-to-head whitespace Passed

Finding ledger:

  • GH-4143658407, P2, rendered-byte quiet-stream allocation: fixed in ee97d050cd04f8213b7372ad4754cff913637122; GitHub thread resolved.
  • IR-SUFFIX-1, P2, logical suffix crossing retained head/tail storage: fixed in this head. Summary and logical-window regressions fail on ee97d05 and pass here. Added ASCII and UTF-8 capture-transition invariants plus a real-command regression.
  • No findings rejected.

The audit followed capture, byte allocation, omission counts, sandbox annotations, command settlement, IPC/protocol validation, programmatic consumers, timeout, cancellation, and cleanup. Result keys and security boundaries are unchanged. GitHub reports the PR conflict-free. A1/A2 are untouched.

The new public native-command regression is blocked locally before execution by the existing storage ancestor guard (/ owned by uid 65534). The guard is unchanged; real-Bash collector probes do not certify SRT admission or confinement.

Not run locally: full code-package suite, live SRT confinement tests, service/API builds or Bun suites. Existing native-sandbox files and README have base-revision full-file Prettier failures; new blocks were formatted without unrelated rewrites. No code-package lint/import-sort task exists.

CI: 10 checks passed at this head, including the full Node 20/22/24 code-package matrix, macOS storage checks, and Linux native-sandbox lane. Only Lambda MicroVM Runner Image (arm64) remains pending; no failure is present.

Independent review completed for a99b89ab7a298575c00374a60968d560d8a73db9: no new findings. Both ledger regressions passed. The reviewer also passed all 12 collector tests, 50,000 deterministic collector probes, and 13 real-Bash collector-boundary checks. Native admission and live SRT confinement were not exercised by that review. Both supported P2 findings are fixed; none were rejected. GitHub head, conflict status, and CI were reconfirmed after review completion.

No merge to main, deployment, or benchmark improvement is claimed.

@danny-avila
danny-avila merged commit e46fca4 into main Oct 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants