Skip to content

fix(cli): render non-ASCII console output on Windows - #384

Closed
lmliheng wants to merge 1 commit into
Tencent:mainfrom
lmliheng:fix/windows-utf8-console
Closed

lmliheng wants to merge 1 commit into
Tencent:mainfrom
lmliheng:fix/windows-utf8-console

Conversation

@lmliheng

@lmliheng lmliheng commented Oct 1, 2026 •

Copy link
Copy Markdown

Why

I drove bsk end to end from PowerShell on Windows 11 + Edge to fill a long,
real-world application form: the campus-recruitment application for WeBank on
Moka (app-tc.mokahr.com/campus-recruitment/webankhr/...), in my own logged-in
profile.

It is the shape of page bsk exists for, and none of it is plain HTML: ~30 fields
across 12 sections, built out of semi-design widgets — sd-Select dropdowns, a
three-level cascader for 籍贯 and 校招面试站点, month/year pickers, a tag-grid
geography picker, autocomplete inputs, hidden input[type=file] for a photo and a
PDF resume, and a two-step submit (预览并提交 → 确认提交). The CLI had to read
page state back, verify thirty fields, and upload two attachments — which is where
this patch came from.

Problems hit

Items 2, 3 and 4 are reported separately, since they are unrelated to each other or
to this fix: #385 (upload permission is not reported up front), #386 (a
session reclaimed mid-use loses the page), #387 (a misplaced global flag fails
on stderr only). Only item 1 is fixed here.

1. Console output was mojibake for every non-ASCII page string — fixed here.

A Windows console starts on its OEM code page (936/GBK here). bsk writes UTF-8,
so every byte that came from the page is decoded against the wrong table. Taken
from a real bsk snapshot of the Moka page — the first line is the captured UTF-8,
the second is what an attached 936 console makes of the same bytes:

- generic "首页" [ref=e7] [cursor:pointer]
- generic "棣栭〉" [ref=e7] [cursor:pointer]

That is not cosmetic. snapshot / observe / evaluate are how an agent reads
page state back, and how it decides whether a custom widget actually committed its
value. With 首页 arriving as 棣栭〉 there is no way to tell a real selection from
a stale label, and matching an option by its expected text silently fails. I worked
around it by forcing UTF-8 on every invocation and reading redirected output as
UTF-8 — a wrapper per command, and one more moving part per step.

2. Attachment upload is blocked by a permission the tool cannot self-diagnose —
not fixed here.

bsk upload failed three ways before giving up:
cdp_failed / "Not allowed" / reason=set_file_input_failed,
then element not visible (no content quads ...), then
the upload trigger did not activate a file input. The root cause is the
extension's "Allow access to file URLs" toggle being off. The error text does
say so, but only after the caller has burned two wrong-target attempts;
bsk doctor does not check it, and the pre-flight check in the skill does not
mention it. effect_state=unknown is correctly marked "do not retry", so this PR
does nothing about it — flagging it because the failure mode costs about ten
minutes every time.

3. A session is recycled while it is still in use — not fixed here.

My first Agent Window session was reclaimed while idle and bsk session list
returned (no active sessions). Moka keeps no server-side draft, so the
twenty-plus fields already filled into the page were gone and the whole form had to
be redone. There is no TTL, no warning, and no way for the caller to hold the
session; from the outside it is indistinguishable from a crash.

4. --session before the subcommand reports nothing useful — not fixed here.

bsk --session <id> click --selector '#x' exits 1 with
unexpected argument '--session' found on stderr only, while the redirected
stdout stays empty. Read from a script, that looks like "the click silently did
nothing", which sends you hunting through the page instead of the command line.
Global flags appear to belong after the subcommand; a one-line usage note or a
non-empty stdout error would have saved that detour.

What changed

  • New crates/bsk-cli/src/cli/console_encoding.rs — a Utf8Console RAII guard.
    enable() reads GetConsoleOutputCP(); if a console is attached and its code
    page is neither 0 nor 65001, it calls SetConsoleOutputCP(65001) and remembers
    the old value, restoring it on Drop so the user's shell is left exactly as
    found. Inert on non-Windows hosts, when neither stdout nor stderr is a terminal,
    and when the console is already UTF-8. Redirected streams are never touched —
    they receive the raw bytes, so no code page is in their path and re-encoding them
    would be a regression.
  • main.rs — binds the guard before Cli::try_parse() so it outlives every
    write, including clap's usage output. #[must_use] on the type keeps the
    lifetime mistake from compiling silently.
  • Cargo.toml — adds the Win32_System_Console feature to the existing
    windows-sys dependency.
  • CHANGELOG.md — entry under [Unreleased] → Fixed.

Five unit tests cover the decision function (redirected_output_is_left_alone,
a_utf8_console_is_left_alone, a_missing_console_is_left_alone,
an_oem_code_page_is_remembered_for_restore, and a guard-inertness check), so the
"do nothing" branches are pinned without needing a console.

Verification

On this machine (Windows 11 Home China, stable-x86_64-pc-windows-gnu):

  • cargo build --release --locked succeeds; the resulting bsk.exe is the binary
    this whole session ran on.
  • cargo test --locked -p bsk --lib console_encoding — 5 passed; 0 failed.
  • cargo fmt --all -- --check — clean.
  • node scripts/check-crate-skill.mjs — clean.
  • cargo clippy --workspace --all-targets --locked -- -D warnings and
    cargo test --workspace --locked fail on this branch, but not because of this
    patch
    . All three clippy findings sit on Windows-only code paths, which is why
    upstream CI stays green (it runs on ubuntu-latest; main's latest run is a
    success): daemon/file_transfer.rs set_private_file takes file: &File whose
    body is #[cfg(unix)], daemon/ipc.rs:793 is a #[cfg(test)] helper, and
    skill_install/harness.rs:339 is inside a #[cfg(windows)] block. A Windows
    clippy -D warnings gate is therefore red on main as it stands, for reasons
    unrelated to this change. cargo test --workspace then aborts while building the
    integration tests with crate ... required to be available in rlib format, but was not found in this form and, on an earlier attempt, failed to mmap ... os error 1455 (page file too small) — a local debug-build resource problem in this
    tree, not a code path. The --lib run above is the part that exercises this
    change.

What is not covered by an end-to-end run. I drove everything from an
automation shell where stdout is always a pipe, so IsTerminal() is false there
and the guard is inert by design — I could not watch an attached console change
code page from that context, and I am not claiming to have done so. The
attached-console path is covered to the extent the split-out decision function can
be: the four restore_code_page cases plus the guard-inertness check, and the
build itself. Confirming the real thing needs one interactive bsk snapshot on a
GBK console, which is a ten-second check for anyone reviewing this on Windows.

Notes

  • This branch is cut from main at 3f10983 and is a single commit.
  • No issue is linked: I looked for the natural home for this and found none, and
    the repo has no CONTRIBUTING or PR template to follow, so this description
    borrows the shape of the existing fix commits. Happy to open an issue and
    rename the branch to fix/<n>-... if that is preferred.
  • The mojibake pair above is not invented: the first line is verbatim from a
    captured snapshot of this page, and the second line is those exact bytes decoded
    as GBK. It is the reason the fix exists.

A Windows console starts on an OEM code page (936/GBK on a Simplified
Chinese system) and reinterprets the UTF-8 bytes bsk prints, so every
non-ASCII character a page contributes reaches the user as mojibake:
`微众银行` prints as `寰紬閾惰`. Observed with `bsk observe` against a
Chinese-language application form in a stock zh-CN console.

Switch the attached console to UTF-8 for the lifetime of the process and
restore the previous code page on drop, so the user's shell is left as
found. Redirected streams are untouched: they receive the raw bytes
verbatim, with no code page in the path.
@lmliheng lmliheng closed this by deleting the head repository Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant