Skip to content

fix(e2e): parse multiline Docker copy sources - #7685

Merged
apurvvkumaria merged 5 commits into
mainfrom
codex/rebuild-openclaw-copy-context
Jul 28, 2026
Merged

fix(e2e): parse multiline Docker copy sources#7685
apurvvkumaria merged 5 commits into
mainfrom
codex/rebuild-openclaw-copy-context

Conversation

@apurvvkumaria

@apurvvkumaria apurvvkumaria commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Rebuild-OpenClaw old-base context discovery now parses backslash-continued Docker COPY instructions as logical instructions and stages every source in multi-source forms. Previously, the physical-line parser omitted four required files from the current Dockerfile.base, causing a deterministic build-context failure rather than runner instability.

Related Issue

Relates to #7140.

Changes

  • Join continued Dockerfile lines while preserving the first physical line for diagnostics.
  • Stage every source operand in supported direct COPY forms while continuing to exclude --from=<stage> copies.
  • Reject unsupported flags, JSON forms, heredocs, alternate escape directives, dangling continuations, and unsafe source paths instead of silently omitting inputs.
  • Add regression coverage for the four previously omitted current-tree inputs, multiline and multi-source forms, CRLF input, stage copies, and malformed forms.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This changes only internal E2E fixture context assembly; no user-facing command, configuration, API, policy, or runtime behavior changes.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Focused independent review confirmed fail-closed parsing, per-source path validation, secret-pattern rejection, stage-copy exclusion, and heredoc handling.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: No documentation files changed; the behavior is confined to the internal test/e2e/ rebuild fixture helper.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — npx vitest run --project e2e-support test/e2e/support/rebuild-openclaw-old-base-context.test.ts (1 file, 5 tests passed)
  • Applicable broad gate passed — Not applicable to this focused two-file E2E helper fix; targeted tests, npm run typecheck, npm run typecheck:cli, project-membership, title-style, source-shape, Biome, and hook checks passed.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved parsing of Dockerfile.base direct COPY instructions, now handling multiline (\-continued) syntax and ignoring comment/blank-line noise.
    • Stricter validation for direct COPY operands and allowed flags: rejects unsupported escape= values, heredoc syntax, JSON-array COPY, invalid/malformed forms, unexpected flags, and multi-stage COPY --from entries.
    • Enhanced safety checks for COPY source paths by blocking unsafe relative tokens and relative paths starting with --.
  • Tests

    • Expanded end-to-end coverage with additional multiline COPY fixtures and more malformed/unsafe cases to ensure failures are explicit.

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria apurvvkumaria self-assigned this Jul 28, 2026
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a78632e3-2ecd-4eaf-9b87-463977e464c6

📥 Commits

Reviewing files that changed from the base of the PR and between ad08ec3 and 6906983.

📒 Files selected for processing (1)
  • test/e2e/support/rebuild-openclaw-old-base-context.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/e2e/support/rebuild-openclaw-old-base-context.test.ts

📝 Walkthrough

Walkthrough

directDockerfileBaseCopySources() now parses continuation-aware Dockerfile.base COPY instructions, validates supported syntax and source paths, skips staged copies, and rejects malformed or unsafe forms. E2E tests cover multiline extraction and additional invalid inputs.

Changes

Dockerfile COPY parsing

Layer / File(s) Summary
Logical COPY parsing and source validation
test/e2e/live/rebuild-openclaw-old-base-context.ts
Adds multiline instruction preprocessing, escape-directive validation, heredoc rejection, supported flag handling, staged-copy skipping, operand validation, and stricter source-path checks.
Multiline and malformed COPY coverage
test/e2e/support/rebuild-openclaw-old-base-context.test.ts
Expands fixtures and assertions for multiline sources, unsupported COPY forms, continuations, and absolute source rejection.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#7646: Expands directDockerfileBaseCopySources() and validateOldBaseContextSource() to parse and validate multiline and grouped Dockerfile.base COPY syntax.

Suggested labels: platform: container

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: parsing multiline Docker COPY sources in e2e context discovery.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/rebuild-openclaw-copy-context

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 6906983 in the codex/rebuild-opencl... branch remains at 96%, unchanged from commit 45da24e in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 6906983 in the codex/rebuild-opencl... branch remains at 81%, unchanged from commit 45da24e in the main branch.

Show a code coverage summary of the most impacted files.
File main 45da24e codex/rebuild-opencl... 6906983 +/-
src/lib/domain/.../connect-env.ts 97% 89% -8%
src/lib/sandbox...rce-identity.ts 88% 88% 0%
src/lib/state/m...-acquisition.ts 75% 75% 0%
src/lib/tunnel/services.ts 73% 73% 0%

Updated July 28, 2026 08:10 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
test/e2e/live/rebuild-openclaw-old-base-context.ts (1)

23-75: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider splitting the flag/operand parsing out of directDockerfileBaseCopySources.

logicalDockerfileInstructions and directDockerfileBaseCopySources together implement a fairly dense hand-rolled parser (continuation joining, escape-directive validation, heredoc detection, leading-flag scanning, --from skip, operand/source validation) inside two long functions. This was already flagged as high complexity by the diff analysis. Extracting the leading-flag scan (lines 93-108) into a small named helper (e.g. parseLeadingCopyFlags(tokens)) would make each piece independently testable and easier to reason about.

As per coding guidelines, "Keep function complexity low; existing complexity hotspots are tracked separately," and this is newly introduced complexity rather than a pre-tracked hotspot.

Also applies to: 77-124

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/rebuild-openclaw-old-base-context.ts` around lines 23 - 75,
Reduce complexity in directDockerfileBaseCopySources by extracting the leading
COPY flag/operand parsing into a small named helper such as
parseLeadingCopyFlags. Have the helper handle flag scanning and --from
detection, while directDockerfileBaseCopySources retains source validation and
copying behavior; keep logicalDockerfileInstructions focused on instruction
joining and validation.

Source: Coding guidelines

test/e2e/support/rebuild-openclaw-old-base-context.test.ts (1)

84-110: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Malformed-forms test doesn't cover all newly added rejections and is loosely asserted.

Two related gaps in this table-driven test:

  • It doesn't include a case for the new escape-directive rejection (# escape= other than \) or the new unsafe-character path rejection ($*?[]"'), even though the PR objectives explicitly call these out as forms that must be rejected.
  • All 8 cases share a single toThrow(/Unsupported direct Dockerfile\.base COPY form|Dangling Dockerfile\.base continuation|Unsupported Dockerfile\.base heredoc instruction/) assertion, so a case could pass for the wrong reason (e.g., an implementation regression that makes case #3 throw the heredoc error instead of the "COPY form" error would go unnoticed).

Consider pairing each input with its specific expected message/regex, and adding the two missing malformed-input cases.

As per path instructions for **/*.test.{ts,js,mts,mjs,cts,cjs}, tests should "flag ... conditionals that make a test pass without exercising its claim," and this shared OR-regex risks exactly that failure mode.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/support/rebuild-openclaw-old-base-context.test.ts` around lines 84 -
110, Expand the unsupportedForms table in the malformed COPY test to include a
non-backslash # escape directive and a COPY path containing unsafe characters
($*?[]"' ). Associate each input with its specific expected error pattern, then
assert each case against that pattern instead of using one shared OR-regex,
preserving distinct messages for unsupported COPY forms, dangling continuations,
and heredoc instructions.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e/live/rebuild-openclaw-old-base-context.ts`:
- Around line 23-75: Reduce complexity in directDockerfileBaseCopySources by
extracting the leading COPY flag/operand parsing into a small named helper such
as parseLeadingCopyFlags. Have the helper handle flag scanning and --from
detection, while directDockerfileBaseCopySources retains source validation and
copying behavior; keep logicalDockerfileInstructions focused on instruction
joining and validation.

In `@test/e2e/support/rebuild-openclaw-old-base-context.test.ts`:
- Around line 84-110: Expand the unsupportedForms table in the malformed COPY
test to include a non-backslash # escape directive and a COPY path containing
unsafe characters ($*?[]"' ). Associate each input with its specific expected
error pattern, then assert each case against that pattern instead of using one
shared OR-regex, preserving distinct messages for unsupported COPY forms,
dangling continuations, and heredoc instructions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8ac50911-b044-42bf-8e19-81149307bf68

📥 Commits

Reviewing files that changed from the base of the PR and between d3eea5a and 7c9f5d6.

📒 Files selected for processing (2)
  • test/e2e/live/rebuild-openclaw-old-base-context.ts
  • test/e2e/support/rebuild-openclaw-old-base-context.test.ts

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · low confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized E2E selections differ; severity counts match.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

2 optional E2E recommendations
  • rebuild-openclaw
  • upgrade-stale-sandbox

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@copy-pr-bot

copy-pr-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/support/rebuild-openclaw-old-base-context.test.ts`:
- Around line 182-186: Update the Dockerfile fixture in the rebuild test to make
the COPY instruction span multiple physical lines while retaining an absolute
source operand such as /etc/passwd. Ensure the test exercises rejection after
continuation parsing, rather than validating only a single-line COPY.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 94f3b361-2c69-4647-bed3-cde0bf7671e9

📥 Commits

Reviewing files that changed from the base of the PR and between 03e1643 and ad08ec3.

📒 Files selected for processing (1)
  • test/e2e/support/rebuild-openclaw-old-base-context.test.ts

Comment thread test/e2e/support/rebuild-openclaw-old-base-context.test.ts
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@cjagwani cjagwani added v0.0.98 and removed v0.0.97 labels Jul 28, 2026
@apurvvkumaria
apurvvkumaria merged commit 7f4b490 into main Jul 28, 2026
71 of 73 checks passed
@apurvvkumaria
apurvvkumaria deleted the codex/rebuild-openclaw-copy-context branch July 28, 2026 18:05
@wscurran wscurran added area: e2e End-to-end tests, nightly failures, or validation infrastructure bug-fix PR fixes a bug or regression labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants