Skip to content

fix(report): write the report card as UTF-8, not the platform locale encoding - #35

Open
dchaudhari7177 wants to merge 1 commit into
bamdadd:mainfrom
dchaudhari7177:fix/utf8-report-output
Open

fix(report): write the report card as UTF-8, not the platform locale encoding#35
dchaudhari7177 wants to merge 1 commit into
bamdadd:mainfrom
dchaudhari7177:fix/utf8-report-output

Conversation

@dchaudhari7177

Copy link
Copy Markdown
Contributor

Found while working on #30 — filing separately rather than folding an unrelated fix into that PR.

Problem

steer-report cannot write a card on Windows. Three tests in tests/test_cli.py fail on a clean main checkout there:

src\steerbench\report.py:799: in build_report
    md_path.write_text(render_markdown(data, dose_png_path.name, layer_png_path.name))
E   UnicodeEncodeError: 'charmap' codec can't encode character 'Δ' in position 1065:
    character maps to <undefined>

The card is not ASCII — the side-effects table header is Δ, the degenerate-trap callouts start with ⚠️, and the prose uses ±. Path.write_text with no encoding= falls back to the platform locale encoding, which is cp1252 on a stock Windows install and cannot encode any of them. CI is green only because Linux runners default to UTF-8.

The same latent mismatch is in the readers: Path.open(newline="") decodes both sweep CSVs and the side-effects CSV in the locale encoding, so a benchmark name written as UTF-8 by the Modal harness would mojibake when the card is rendered on a non-UTF-8 machine.

Change

Name the encoding at every text I/O site in src/steerbench — five call sites, all one-liners:

file site why
report.py md_path.write_text / html_path.write_text the actual crash
report.py _read_sweep_rows / load_side_effects CSV opens decode CSVs the same way whoever wrote them encoded them
cli.py _ensure_side_csv stub write consistency; the stub is read straight back
metrics.py MMLU / GSM8K slice cache read + write the cache is JSON, which is UTF-8 by spec

No behaviour change on a UTF-8 platform; the bytes written are identical. The HTML already emits <meta charset="utf-8">, so writing it in the locale encoding was self-contradictory.

experiments/modal_app.py is left alone deliberately: it runs only inside the Modal Linux containers, and it is outside the lint/type gates (extend-exclude).

Test

test_report_files_are_written_as_utf8 in tests/test_report.py.

Asserting on the written bytes alone would pass with or without the fix on a UTF-8 runner, so the test instead runs the whole CSV-in / card-out path in a fresh interpreter under -X warn_default_encoding. In that mode io.text_encoding() reports every call that relies on the default encoding as an EncodingWarning attributed to its caller, so unencoded I/O is detectable on every platform, not just where it crashes. Only warnings whose file lives inside the installed package count — optional third-party dependencies (matplotlib's font manager raises one) are not this repo's problem, and the first version of this test caught that instead, which is what led to the narrower filter.

It also reads both artefacts back as UTF-8 and asserts the Δ and ⚠️ survive.

Reverting just the src/ half makes it fail and name all five sites:

AssertionError: text I/O without an explicit encoding:
report.py:211, report.py:211, report.py:276, report.py:799, report.py:800

I followed the existing test_no_matplotlib_imported fresh-interpreter pattern rather than inventing a new one.

Checks

ruff check ., ruff format --check ., mypy src, pytest -q all pass — and on Windows the full suite is now green under the default cp1252 locale, where it was 2 failed before this change (3 with the new test).

One unrelated local note: mypy src reports a syntax error inside numpy/__init__.pyi if numpy is present in the environment (it arrives with the optional report extra, not with the dev group) because the config targets python_version = "3.11". mypy --python-version 3.12 src is clean, and CI does not install numpy.

…encoding

build_report crashed on Windows: the card carries non-ASCII (the delta column
header in the side-effects table, the warning sign on degenerate-trap
callouts), and Path.write_text with no encoding= falls back to the locale
encoding, which is cp1252 on a stock install and cannot encode either
character. tests/test_cli.py's three rendering tests fail there on a clean
checkout with UnicodeEncodeError; they pass on CI only because Linux defaults
to UTF-8.

Name the encoding at every text I/O site in the package: the two card writes,
both CSV readers, the side-effects stub, and the MMLU/GSM8K slice cache. The
HTML already declares charset=utf-8, so writing it in anything else was
self-contradictory.

The regression test runs the whole CSV-in / card-out path in a fresh
interpreter under -X warn_default_encoding and fails if any EncodingWarning is
attributed to a file inside the package, so it catches a regression on UTF-8
platforms too rather than only where the crash happens. Reverting the fix makes
it fail and name all five call sites.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant