Skip to content

Align compiler with gh-aw-firewall v0.28.6 (Cloud Hypervisor filesystem.allowWrite support) #55169

Description

@lpcox

Summary

gh-aw-firewall v0.28.6 ships full filesystem.allowWrite support for the Cloud Hypervisor microVM runtime. The compiler-side plumbing for allowWrite already exists in this repo but is inert by default, because DefaultFirewallVersion is still below the gate. This issue tracks aligning the compiler.

⚠️ This is not a simple version bump. Activating the gate as-is would very likely regress every Cloud Hypervisor workflow. See Blocker 1 below — please read before bumping.

Merged upstream PRs

Firewall PR Merge commit Content
#7660 6aab9a15 allowWrite planner (landed inert)
#7661 62de6370 Host mount-tree enforcement (landed inert)
#7669 664ffc1c Cloud Hypervisor runtime integration — makes the above live
#7672 4017c61c Critical fix — overlay mount propagation
#7670 5853b08b .awf-home prerequisite docs

All five are contained in tag v0.28.6 (fe3bcc3a801091efda4fb4a7902078a8fd0a9dfb).


Current compiler state

Already present, no changes needed to the mechanism itself:

  • pkg/constants/version_constants.go:134AWFFilesystemAllowWriteMinVersion = "v0.28.5"
  • pkg/workflow/awf_feature_flags.go:93awfSupportsFilesystemAllowWrite() gate
  • pkg/workflow/awf_config.go:548 — emits the filesystem section when gated, logs a skip otherwise

Blocking the feature:

  • pkg/constants/version_constants.go:67DefaultFirewallVersion = "v0.28.4" → below v0.28.5, so the gate never opens and the filesystem section is never emitted.

🔴 Blocker 1 — the default write path makes the CH workspace read-only

pkg/workflow/sandbox.go:286 (ensureDefaultAgentWritePath) unconditionally appends defaultAgentWorkspaceWritePath (/tmp/gh-aw/agent) to filesystem.allowWrite for every AWF sandbox config, regardless of runtime.

Under Docker/gVisor that is fine. Under Cloud Hypervisor it is not, because /workspace and /tmp/gh-aw are separate exports and the planner narrows each export independently. Since no allowed path falls under /workspace, that export gets narrowed to nothing.

I verified this empirically against the released v0.28.6 planner, passing exactly what ensureDefaultAgentWritePath produces today (allowWrite: ["/tmp/gh-aw/agent"]):

TAG=workspace   disposition=read-only  hostRoot=ro  guest=ro  overlays=0
TAG=tmp-gh-aw   disposition=selective  hostRoot=ro  guest=rw  overlays=1

Consequences if the version is bumped without addressing this:

  1. /workspace becomes fully read-only — the repo checkout is no longer writable, so the agent cannot edit files, and git operations that write to the worktree or .git fail.
  2. HOME becomes read-only — Cloud Hypervisor sets the guest home to /workspace/.awf-home, which is inside the now-read-only workspace export.

This is a silent, wide-blast-radius behavior change: it activates on the version bump alone, with no frontmatter change by any workflow author.

Suggested resolution — make the defaults runtime-aware. When the runtime is Cloud Hypervisor, ensureDefaultAgentWritePath should also seed the workspace paths the agent genuinely needs, at minimum:

  • /workspace (or a narrower set if the intent is to genuinely restrict the checkout), and
  • /workspace/.awf-home

Alternatively, skip emitting filesystem entirely for Cloud Hypervisor until the default set has been designed deliberately. Please treat "what should be writable by default under CH" as a product decision rather than inheriting the Docker default by accident.


🔴 Blocker 2 — .awf-home must exist on the host before AWF starts

The planner requires every allowWrite path to already exist, and Cloud Hypervisor does not create $GITHUB_WORKSPACE/.awf-home before planning. (src/microvm/workspace.ts does create it, but the CH backend does not use that module.)

So if /workspace/.awf-home is added to allowWrite without the host directory existing, planning fails closed with:

... not an existing path within a writable Cloud Hypervisor export: /workspace/.awf-home

The compiler must emit a mkdir -p "$GITHUB_WORKSPACE/.awf-home" step before the AWF invocation whenever it includes that path. This is documented as a prerequisite in docs/cloud-hypervisor-foundation.md and docs/awf-config-spec.md upstream — deliberately not auto-created, to preserve the existing-path/narrowing semantics.

Worth also confirming that /tmp/gh-aw/agent itself is created host-side before AWF starts; I could not find a compiler-generated mkdir for it, and the same fail-closed rule applies.


🟠 Blocker 3 — v0.28.5 is not a safe minimum for Cloud Hypervisor

AWFFilesystemAllowWriteMinVersion is v0.28.5, but selective allowWrite was broken on every real host until v0.28.6.

Firewall #7672 fixed a mount-propagation defect: a bind mount joins its source's peer group, so on any systemd host (including GitHub-hosted runners) every overlay arrived shared:N and the fail-closed propagation assertion aborted the run:

Staged mount tree propagation would leak: .../0-workspace/allowed has shared:1

This was caught only by live KVM CI; unit tests missed it because the test fake modelled mount --bind as producing an already-private mount.

Therefore: pinning >= v0.28.5 would advertise support for a version where CH allowWrite reliably fails. Please either raise AWFFilesystemAllowWriteMinVersion to v0.28.6, or introduce a separate CH-specific constant (e.g. AWFCloudHypervisorFilesystemAllowWriteMinVersion = "v0.28.6") so Docker/gVisor keep the lower bound while CH requires v0.28.6.


🟡 Stale schema description

pkg/workflow/schemas/awf-config.schema.json still carries the pre-v0.28.6 support statement:

"This option is currently supported by Docker and gVisor compose runtimes."

Released v0.28.6 says:

"This option is supported by the Docker and gVisor compose runtimes and by the Cloud Hypervisor microVM runtime; it is rejected with the sbx runtime and with Docker-in-Docker agent execution."

Re-sync the embedded schema from the released one, and grep for any other prose in docs/ asserting that allowWrite is unsupported under Cloud Hypervisor.


Proposed checklist

  • Decide the Cloud Hypervisor default allowWrite set and make ensureDefaultAgentWritePath runtime-aware (Blocker 1)
  • Emit mkdir -p for .awf-home (and confirm /tmp/gh-aw/agent) before the AWF invocation (Blocker 2)
  • Raise the allowWrite min version to v0.28.6, or add a CH-specific constant (Blocker 3)
  • Bump DefaultFirewallVersion v0.28.4v0.28.6
  • Re-sync pkg/workflow/schemas/awf-config.schema.json from released v0.28.6
  • Refresh embedded container digest pins
  • Regenerate pinned .lock.yml workflow artifacts
  • Add a .changeset/ patch entry (house style: "Bump the default gh-aw-firewall version to vX.Y.Z and refresh embedded container digest pins.")
  • Add compiler tests covering CH + allowWrite emission, the runtime-aware defaults, and the version gate boundaries

Suggested validation

Because the failure mode in Blocker 3 was invisible to unit tests, please validate at least one live Cloud Hypervisor run end-to-end rather than relying on generated-YAML assertions alone. Upstream, the equivalent coverage is the label-gated cloud-hypervisor-kvm job (~25 min).


Filed from the gh-aw-firewall side after landing #7669 / #7672 / #7670. Findings above were verified against the released v0.28.6 planner and the current gh-aw main. Happy to help review the compiler change.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions