Skip to content

Align Environment Files create and list with official hosted responses - #49

Merged
SaladDay merged 11 commits into
mainfrom
codex/environment-files-wire
Sep 23, 2026
Merged

SaladDay merged 11 commits into
mainfrom
codex/environment-files-wire

Conversation

@SaladDay

@SaladDay SaladDay commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Environment Files create and list now match the first owned official hosted observations: status codes, envelope, query handling, empty pages for non-listable paths, path validation and error fields. File-access safety is unchanged. The pinned baseline is unchanged (SDK 3.13.0 / d7c41ef / agents=v1).

Behavior

  • Create: returns 201 with the same four-field body.
  • List envelope: {object: "page", data, next, has_more}, where has_more is true exactly when next is set.
  • List query:
    • Unknown keys are ignored, as on the shared lists.
    • A repeated key returns the Beta duplicate-field error.
    • Malformed query encoding is still rejected locally; this is documented.
  • Non-listable paths:
    • A missing directory, a regular file, or a symlink to a directory lists as an empty page. Links are never followed or opened.
    • A new native directory-read result, not_directory, keeps these cases apart from real failures. It is reported only when walking the requested path below an intact workspace root fails with "missing" or "not a directory".
    • A root that is removed or replaced, before or during the read, is detected by comparing device and inode with the held descriptor, and stays 404. This works on overlayfs, and bind mounts give no false positives.
    • Permission, I/O, transport and uncertain results keep their errors.
    • Daemons without a local workspace binding (the Claude SDK reader) keep 404/503, as documented.
  • Path validation:
    • Non-clean paths (a trailing slash, //, ., ..) are rejected rather than normalized.
    • List and create validation errors use code invalid_request_error with the observed official messages.
    • Unknown create fields report Unknown parameter: '<field>' with the field as param, but only for valid JSON and names that are at most 256 printable bytes. Other names get a generic, non-echoing invalid_request_error, so responses stay bounded.
  • Provisioning: Files on a still-provisioning openai_hosted Environment return the observed 400 instead of 503. The check runs after the tenant lookup and validation, and before any source read or execution.

Evidence

Campaign scan 2 hosted findings HE-10/16/18/32/34–39 (3 owned official hosted Sessions, all deleted). Recorded in contracts/agents-api/environment-files.md and operation-evidence.md rows 27–28.

Validation

  • Live acceptance through real Core, the daemon and a Codex Runtime on Core-managed Docker openai_hosted, with no model Turns. It was built from the exact source, and the daemon and Rust helper hashes were verified inside the running Runtime.

    • On main, all nine rows (F1–F9) failed. On the final runtime (c275bad), all passed.
    • The provisioning window was actually hit: the Environment was still pending 11 ms after creation.
    • An over-long unknown field gets a 118-byte non-echoing 400.
    • Tenant B gets 404 everywhere.
    • The last commit only changes API error selection and is covered by handler tests and the gate.
  • Rust: tests for every link kind, root removal and replacement (they fail with the check disabled), plus fmt and clippy. The reviewer additionally probed overlayfs and bind mounts under an unprivileged namespace.

  • Go: handler, Worker (PostgreSQL), gateway and daemon tests, including a native helper confinement test.

  • TS: client and Web unit tests for the strict envelope and the 201.

  • Server gate on this head: make -o check-web check plus Web typecheck, core-doctor, the client and Web unit tests and the build all pass. The Playwright browser cases were not run on the server (no Google Chrome; skip approved by the user).

  • Generated files: make openapi is byte-identical after the rebase onto Ship a zero-node Core installer with opt-in local sandboxes #48.

  • Reviews: two fresh full-diff blind reviews and one focused safety review by Claude Code subagents (the user-approved replacement for GPT-6 Astra). All in-scope findings are fixed:

    • an unbounded echo amplification;
    • a root removed mid-walk shown as an empty page, including on overlayfs;
    • the unknown-field code;
    • malformed-JSON and U+FFFD classification.

    The final small fix was verified by tests and the gate without a new review, per the user's rule.

Deferred

  • Recursive listing.
  • Parent creation, no-overwrite and the 5 MiB limit (these change the shared installer).
  • The Environment files[] projection.
  • environment.ready events.
  • Malformed query encoding tolerance.
  • The duplicate-key report order.
  • An ENAMETOOLONG component returning 503.
  • NFS qualification.
  • A namespace test for the device half of the identity check.

No full protocol compatibility is claimed.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.

The official Files.list returns an empty page when path names a missing
directory, a regular file or a symlink to a directory (HE-36, HE-37). Core
returned 404 for a missing path and 503 for the others.

The native directory helper now reports a distinct not_directory result only
when walking the requested path below the anchored root fails with ENOENT or
ENOTDIR. Components are opened with O_PATH|O_DIRECTORY|O_NOFOLLOW, so a link
fails at its own component and no target is followed, opened or listed. Every
component is validated before any is opened, so an invalid request cannot
become an empty listing. The daemon and gateway carry the code only for
directory reads, and Core maps it to an empty directory after the same
confirmed release that a listing requires.

Real failures keep their errors: invalid_path still covers unsafe entry names,
and not_found still covers a missing workspace root and entries removed during
observation, so neither is mapped. The shared workspace path code and the write
installer are unchanged.

Rust, daemon, gateway and real-PostgreSQL Worker tests cover the classification,
links inside and outside the workspace, dangling links, a replaced root, release
confirmation and the unchanged rejections.
…al service

Files.create now returns 201 with the same four fields (HE-10). Files.list
returns the official token page: object "page", data, next and has_more, which
is true exactly when next is set (HE-32). Paging and ordering are unchanged.

The list keeps its own path and cursor parsing but follows the shared list
rules: unknown query keys are ignored (HE-34) and a repeated path, limit, order
or page returns the Beta duplicate-field error (HE-35). The path must already be
in cleaned form; a trailing or repeated separator, "." or ".." now returns the
observed "non-reserved directory" error instead of being normalized (HE-38).
Relative and outside paths and every rejected page token use the observed
messages with code invalid_request_error (HE-39). Files.create reports the
observed path messages, keeping the official environment.files[0].path field
name, and names the first unknown body field as param (HE-16). The accepted
create paths are unchanged; unsampled body errors keep the local code. The
Environment lookup still runs first, so foreign Environments equal missing ones.

The TypeScript client accepts only the new envelope and 201, and Core Web sends
the cleaned form of a directory typed with a trailing slash. The Web e2e fixture
serves the same shapes. OpenAPI is regenerated.
…oning

The official service rejects Files.create and Files.list on a hosted
Environment that has not connected yet with 400 invalid_request_error, param
null, "the hosted environment is still provisioning; wait until it is connected
before accessing files" (HE-18). Core instead attempted execution and could
return 503.

After the tenant-scoped lookup and request validation, both operations now
return that error when the stored Environment type is openai_hosted and its
status is pending. The check precedes source-file resolution and any execution
access, so a foreign Environment still equals a missing one and nothing is read
or written. Connected and disconnected hosted Environments and every
self_hosted status keep the existing execution path.
environment-files.md gains a dated section with the F1-F9 matrix, the official
request IDs from the hosted-environment scan (HE-10, 16, 18, 32, 34-39), the
decisions on native classification, path messages, token errors and the
provisioning check, and the deferred items. The operation evidence inventory
registers the batch as G and updates rows 27 and 28. CONTRIBUTING, the contract
README, the list query and validation error sections and the Web protocol
coverage no longer describe the strict Files.list key parser, path normalization
or the old page projection.

The opt-in official scripts now require the exact page envelope on raw pages,
expect 201 from raw Files.create, and gain helpers that replay the list rows
(unknown and repeated keys, missing, file and symlink empty pages, path and
token errors), the create rows (path and unknown-field errors, 201) and the
pending hosted check through raw HTTP and the pinned SDK. Foreign-tenant checks
also cover create bodies, repeated keys and missing paths.
The unknown-parameter error repeated the caller's JSON key in both message and
param, and JSON escaping grows "<" and ">" to six bytes each. A 4 MiB key
produced a 48 MiB response, and a create body can approach 67 MiB.

The observed error now names the field only when it is at most 256 bytes of
valid, printable UTF-8. Any other unknown key gets the existing local generic
400 without echoing it. Tests cover an HTML-heavy name that is echoed, a
257-byte and a 4 MiB name, tab, line-separator and control-character names,
and a printable non-ASCII name, each with a response under 4 KiB and no write.
The other errors this batch added use fixed messages or a key from the fixed
supported set.
If the anchored workspace root was deleted after the helper opened it and
before the requested path was walked, the first lookup failed with ENOENT and
was reported as not_directory, an empty page. It was 404 before this batch.

Before reporting not_directory, the helper now checks with fstat that the
opened root still has a link. A removed root keeps not_found, and so 404. A
Rust test removes the root between opening and walking. It checks missing,
existing and nested paths and the root listing, and fails without the check.
A daemon without a local workspace binding lists directories through the
unchanged Claude SDK adapter reader. There F5 and F6 keep 404 for a missing path
and 503 for a regular file or symlink. Only local workspace readers return the
empty page.

The list description and regenerated OpenAPI, environment-files.md, the
contract README and the operation evidence row now state this exception. The
native official script accepts directory_reader "claude_sdk_adapter" for
claude_sdk and then expects the documented 404/503 instead of empty pages. The
default remains a local workspace binding.
The Environment Files list ignores well-formed unknown keys like the shared
lists. It still rejects the whole request on malformed query encoding such as
"?foo=%GG" or ";" separators, where the shared lists drop those pairs. The list
description and regenerated OpenAPI, environment-files.md, CONTRIBUTING, the
list query and validation error sections and the operation evidence row now
say so. Behavior is unchanged; a handler test adds the ";" separator case next
to the existing malformed percent-encoding case.
The removed-root check relied on st_nlink being zero. On overlayfs, a
directory removed from an image layer can keep st_nlink 1, so after rm -rf of
the root a failed walk still became not_directory and an empty page instead of
404.

Before reporting not_directory, the helper now reopens the root path without
following links and compares device and inode with the held root descriptor.
A removed root, a root replaced by another directory and a root replaced by a
symlink keep not_found (404). The same check runs after an empty listing,
because reading a removed directory ends like an empty one. That also closes
the earlier edge where removing the root between opening the default path and
reading it produced a 200 empty page. Entries still read through the held root
are returned as before.

Rust tests cover a root removed after opening (the default, existing, missing
and nested paths, and the empty read of the removed root), a root replaced by
another directory or a symlink, and an unchanged root that still lists missing
paths and empty directories. The tests fail with the check disabled.
…choed

An unknown body field whose name is over 256 bytes or not printable UTF-8 is
still not echoed, but it now uses code invalid_request_error with param null and
the message "Unknown parameter.", matching the F8 code, instead of the local
invalid_request error. The tests and environment-files.md are updated.
The unknown-field scan ran before the body was known to be valid JSON, so
'{"foo":1,' reported "Unknown parameter: 'foo'." while '{"type":"inline",'
got the malformed-body error. The scan now skips bodies that are not valid
JSON, and they get the existing malformed-body error.

encoding/json replaces invalid UTF-8 and lone surrogates with U+FFFD before the
name is checked, so the UTF-8 guard never fired and '{"\xff\xfe":1}' echoed
U+FFFD. A name containing U+FFFD is no longer echoed and uses the
invalid_request_error "Unknown parameter." error instead.

Tests cover truncated and trailing-data bodies with an unknown first key,
invalid bytes, a lone surrogate and a literal U+FFFD. environment-files.md now
says the root identity check relies on local filesystem inode pinning and that
network filesystems such as NFS are not qualified.
@SaladDay
SaladDay merged commit 4f3822a into main Sep 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant