feat(fs): land remaining fs-epic children onto main (#978, #979, #977, #984, #973, #975, #974)#995
Merged
Merged
Conversation
…ess (#978) Introduce BufferEncoding — one BCL-only source of truth for Node encoding conversions (utf8/utf-8, ascii, latin1/binary, base64, base64url, hex, utf16le/ucs2) used by both Buffer and fs in interpreter mode, so a given encoding name behaves identically across them. Adds base64url (previously threw in interp) and a ToBytes(data, encoding) that yields raw bytes from a Buffer/TypedArray verbatim or a string via the encoding. Interp fs read/write/append (sync + async) now: - write Buffer/TypedArray data byte-exact (no more .ToString(), which corrupted binary writes) and honor the encoding for string data; - decode reads with the requested encoding (was always UTF-8), returning a Buffer when no encoding is given. SharpTSBuffer's encode/decode delegate to BufferEncoding. The fs.ts facade forwards the options arg to write/append; the primitive write/append types widen to (path, data, options?). Interp half of #978: binary round-trips with 0x00/0xFF byte-identical and each encoding is correct. 548 fs+Buffer tests green (both modes). The compiled IL twin (emitted encoder + fs write-bytes + $Buffer utf16le/base64url parity) follows next so compiled matches.
Bring compiled-mode Buffer and fs encoding to byte-for-byte parity with the
interpreter (BufferEncoding), staying BCL-only/standalone.
$Buffer encoder (RuntimeEmitter.TSBufferInstance/Static): the toString/from
encoding dispatch only handled utf8/ascii/base64/hex and DEFAULTED everything
else to UTF8 — so utf16le/ucs2 silently returned the wrong string and base64url
was unsupported. Added latin1/binary (Encoding.Latin1), utf16le/ucs2/utf-16le
(Encoding.Unicode), and base64url (strip '='/map +/ on decode; map -/_ and
re-pad on encode). Fixes a real Buffer interp<->compiled divergence.
Compiled fs (RuntimeEmitter.FsSync + FsModuleEmitter + Fs{Async,Wrappers}):
write/append now persist RAW BYTES via a new BCL-only FsToBytes helper ($Buffer
bytes verbatim, else encode the string via the corrected $Buffer encoder) and
File.Write/AppendAllBytes — no more Stringify (which corrupted binary writes).
read decodes via the $Buffer encoder when an encoding is given, else returns a
$Buffer. A FsEncodingName helper extracts the encoding from a string-or-options
value; the write/append helpers and emitter dispatch thread the options arg.
buf-enc.ts and fs-enc.ts now produce identical output compiled vs interpreter;
compiled fs programs remain standalone (no SharpTS.dll). +3 dual-mode parity
tests (binary round-trip, hex/base64/base64url, latin1/utf16le). Full xUnit
suite green (14430/0).
…tants (#979) mkdirSync now follows Node semantics (implemented in the fs.ts facade over existsSync + the raw recursive create, so both modes share one impl, no IL): non-recursive throws EEXIST on an existing path and ENOENT on a missing parent; recursive returns the first (shallowest) directory created, or undefined when nothing was created. accessSync/access enforce the mode argument: W_OK on a read-only file throws EACCES (best-effort — read-only attribute on Windows, shared CheckAccess for the sync + async interp paths; the compiled FsAccessSync gained the same check with a null-safe mode read so the async access(path) path that forwards undefined no longer crashes). F_OK still yields ENOENT on a missing target. fs.constants is completed in the facade (one table shared by fs.constants and fs.promises.constants): permission bits S_IRWXU/S_IRUSR/.../S_IXOTH, O_DIRECTORY/ O_NOFOLLOW/O_SYNC/O_DSYNC/O_NONBLOCK/O_NOATIME/O_NOCTTY, and UV_FS_O_FILEMAP/ UV_FS_SYMLINK_DIR/JUNCTION, keeping the existing values openSync flag parsing uses. +4 dual-mode parity tests. Full xUnit 14438/0; TypeScript conformance unchanged. Note: the facade helper had to avoid the name __dirname (a CommonJS-style global string in SharpTS). Deferred (issue minor-completeness, overlaps #975): sync Dir.readSync/closeSync + callback opendir. Separately found (not fixed here): the compiled mkdtempSync doubles an absolute prefix (prepends GetTempPath) — interp is correct; a follow-up.
Stats was incomplete and inconsistent — sync returned {isFile,isDirectory,size}
only, async added times/*Ms, compiled used 0.0 placeholders, and dev/ino/mode/
predicates were missing or hardcoded false. Replace all of it with Approach B:
- New statRaw/lstatRaw/fstatRaw primitives (interp C# + compiled BCL-only IL,
RuntimeEmitter.FsStatRaw.cs) return ONE flat numeric record: mode (type bits
| perm bits, from File.GetUnixFileMode on Unix / 0o666·0o777±readonly on
Windows), size, atime/mtime/ctime/birthtimeMs, and documented fallbacks for
dev/ino/nlink/uid/gid/rdev/blksize/blocks. fstat recovers the path via
FileStream.Name. The promise stat/lstat primitives now return the same record.
- A single TS `Stats` class in the facade (built by one `__makeStats` factory)
derives all seven is*() from mode & S_IFMT, exposes Date getters from *Ms, the
full field set, and a { bigint: true } variant (*Ns + BigInt fields). Every
stat path — statSync/lstatSync/fstatSync, callback stat/lstat, fs.promises —
goes through it, so sync and async are identical by construction.
- Dirent gains parentPath/path (Node 20+) and uniform is*() method predicates.
statSync(p) and await stat(p) now return byte-identical shape and values across
interp and compiled. Standalone preserved. +6 dual-mode parity tests; full xUnit
14444/0; TypeScript conformance unchanged.
) The emitted FsMkdtempSync IL computed Path.GetRandomFileName() but discarded it, stored the prefix into the "tempFileName" local, then did String.Concat(prefix, prefix) — so mkdtempSync(os.tmpdir() + '/foo-') produced a doubled path like '…/Temp/foo-…/Temp/foo-' and threw EACCES. The interpreter was correct. Rewrite the IL to form `Path.Combine(Path.GetTempPath(), prefix + random)`, mirroring the interpreter. Path.Combine returns the suffix verbatim when it is rooted, so the canonical mkdtempSync(path.join(os.tmpdir(), 'foo-')) yields '…/foo-XXXXXX' in both modes (byte-identical) instead of crashing. +1 dual-mode regression test. Full xUnit 14446/0; TypeScript conformance unchanged.
Implements the modern recursive remove/copy APIs as pure TS in the facade over
the existing primitives, so interpreter and compiled share one implementation
(Approach B):
- rmSync(path, { recursive, force }): removes a file or (recursively) a tree;
force makes a missing path a no-op, a directory without recursive throws
ERR_FS_EISDIR. Reuses the primitive rmdirSync(recursive) for the tree delete.
- cpSync(src, dest, { recursive, force, errorOnExist, dereference,
preserveTimestamps, filter }): recursive copy over stat/readdir/mkdir/copyFile,
with a sync filter predicate, ERR_FS_EISDIR without recursive, and
ERR_FS_CP_EEXIST under errorOnExist.
- Callback rm/cp and promises.rm/promises.cp wrap the sync walk in a Promise, so
sync and async behave identically. fs/promises.ts routes through the shared
facade impl.
Verified byte-identical interp↔compiled across recursive remove, force/ENOENT,
recursive copy, errorOnExist, and a filter. +4 dual-mode tests; full xUnit
14450/0; TypeScript conformance unchanged.
Note: rmSync originally used try/catch with an early return in the catch, which
triggered a compiled-mode InvalidProgramException (could not be minimally
reproduced); rewritten to use existsSync, which both avoids it and is simpler.
- promises.opendir + opendirSync now return a Dir handle implemented in the
facade over readdirSync({ withFileTypes: true }): path, read()/readSync()
(Dirent | null), close()/closeSync(), and [Symbol.asyncIterator]() — so
`for await (const ent of dir)` enumerates entries then ends, both modes.
- promises.watch(filename, options?) returns a pull-based async iterator that
bridges the FSWatcher's 'change' stream: events arriving between pulls are
queued, a pull with no event parks a resolver. AbortSignal / iterator return()
close the watcher and end iteration so the event loop drains.
Both are pure facade TS, byte-identical interp↔compiled for the deterministic
paths. +2 dual-mode tests (opendir async iteration; watch iterator + abort
termination). Full xUnit 14454/0; TypeScript conformance unchanged.
Known limitation (filed as #985): compiled AbortSignal has no working abort
callback (addEventListener throws from facade code; onabort never fires), so in
compiled mode a parked watch iterator only observes abort on the next pull (the
facade polls signal.aborted), not immediately. The interpreter terminates
promptly. The watch test is deterministic to avoid FSWatcher timing flakiness;
live change-event observation is verified manually.
…/fstat/ftruncate) (#974) All implemented as thin TS wrappers over the *Sync primitives, sharing the callback/promise plumbing (Approach B), so interpreter and compiled get one implementation: - callback chown/lchown + promises.chown/lchown wrap chownSync/lchownSync. - callback open(...,cb) -> (err, fd), close(fd, cb), read(fd, buf, off, len, pos, cb) -> (err, bytesRead, buf), write(fd, buf, off, len, pos, cb) -> (err, bytesWritten, buf), fstat(fd, opts?, cb) -> (err, Stats), ftruncate(fd, len?, cb). A shared __promisifyValue defers the callback off the caller's frame and carries err.code (e.g. EBADF) through. Happy-path fd round trips (open -> read at a position -> write -> fstat -> ftruncate -> close) are byte-identical interp<->compiled. +4 dual-mode tests. Full xUnit 14458/0; TypeScript conformance unchanged. Note: a stale fd / unsupported chown surface different error CODES across modes (compiled EINVAL vs interp EBADF/ENOSYS) — a pre-existing divergence in the sync fd primitives that these async wrappers faithfully propagate; filed as #986.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recovery PR — lands the rest of the fs epic onto
mainThe 8 stacked PRs (#987–#994) were each merged into their parent feature branch rather than
main, so only #969 (PR #987) actually reachedmain; the other seven children ended up stranded in the (now-merged) stacked branches.This branch (
wrk/issue-974-async-chown-fd) is the top of the original stack and contains all of them linearly. Sincemainalready has the #969 commits, this PR's diff is exactly the remaining seven children:fs.constants(mkdir options, access mode, O_*/S_* bits) #979 — mkdir semantics + accessSync mode + completefs.constantsStats/Direntobjects (sync↔async consistency) #977 — unify Stats/Dirent via one TS Stats class over statRawmkdtempSyncabsolute-prefix fixrm/rmSync+cp/cpSync/promises.cp#973 — rm/rmSync + cp/cpSync/promises.cpwatch(async iterator) +opendir#975 — promises.watch (async iterator) + opendir/Dirchown/lchown+ callback fd ops (open/read/write/close/fstat/ftruncate) #974 — async chown/lchown + callback fd opsAll verified green together at this tip: full xUnit 14458/0, TypeScript conformance baseline unchanged, Test262 only the pre-existing #882-family failures.
Closes #978, #979, #977, #984, #973, #975, #974.