Skip to content

fix: UTF-8 result writes, and make --force actually discard prior results - #22

Open
dat999zx wants to merge 1 commit into
HUST-AI-HYZ:mainfrom
dat999zx:fix/utf8-and-force
Open

fix: UTF-8 result writes, and make --force actually discard prior results#22
dat999zx wants to merge 1 commit into
HUST-AI-HYZ:mainfrom
dat999zx:fix/utf8-and-force

Conversation

@dat999zx

@dat999zx dat999zx commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Two independent bugs, both silent, both hit outside a UTF-8 default locale. Neither is specific to any one method — they sit on paths every method goes through.

1. Result writes assume a UTF-8 default encoding

open(path, "w") with no encoding uses the platform default, which is cp1252 on a stock Windows install. The first non-ASCII value in a retrieved context or a saved result raises UnicodeEncodeError and kills the run.

FactConsolidation reaches this immediately — pesäpallo is one of the gold answers — but the writes are in agent.py and main.py, so every method crashes, not just one.

Five call sites now pass encoding="utf-8". The two that dump retrieved context and final results also pass ensure_ascii=False, so non-ASCII stays readable in the JSON instead of being escaped.

2. --force did not discard prior results

--force is documented as "Force re-run even if results already exist", but it only reached should_skip_context. load_existing_results ran unconditionally, so a forced run still:

  • seeded metrics from the saved rows, averaging old scores into the new number, and
  • restored the query high-water mark, so should_skip_query — which never took the flag — skipped the very queries --force asked to re-run.

Re-running after fixing your method therefore reports a figure partly produced by the old code, with nothing in the output to indicate it happened.

We hit this. A broken 5-question run left rows behind; the next 100-question run reported 84%, but rows 0–4 still carried input_len 25373/20374/16393/6468/20727 against 108–146 for the rest. Over the 95 clean rows it was 86.3%; the 5 stale rows scored 2/5.

load_existing_results now takes force_rerun and returns empty state when set. Resume behaviour without the flag is unchanged.

Verification

With a saved five-row results file:

rows reloaded resumes at query
without --force 5 5
with --force 0 0

Note on scope

Kept deliberately to these two fixes so they can be reviewed quickly and independently. Happy to split them into separate PRs if you'd prefer.

input_len turned out to be a useful contamination check while debugging this — rows produced by a different code path carry a visibly different context size, so a jump in it across rows of one results file means the rows are mixed.

…ults

Two independent bugs, both hit on any platform whose default encoding is not
UTF-8, and both silent.

1. Result writes assumed a UTF-8 default encoding

   open(path, "w") with no encoding uses the platform default, which is cp1252
   on a stock Windows install. The first non-ASCII value in a retrieved context
   or a saved result then raises UnicodeEncodeError and kills the run.

   FactConsolidation reaches this immediately -- "pesäpallo" is one of the gold
   answers -- but nothing here is method-specific. The writes are in agent.py
   and main.py, on paths every method goes through, so every method crashes.

   Five call sites now pass encoding="utf-8". The two that dump retrieved
   context and final results also pass ensure_ascii=False, so non-ASCII stays
   readable in the JSON rather than being escaped.

2. --force did not discard prior results

   The flag is documented as "Force re-run even if results already exist", but
   it only reached should_skip_context. load_existing_results ran
   unconditionally, so a forced run still:

     - seeded metrics from the saved rows, averaging the old scores into the
       new number, and
     - restored the query high-water mark, so should_skip_query -- which never
       took the flag -- skipped the very queries --force asked to re-run.

   Re-running after fixing your method therefore reports a figure partly
   produced by the old code, with nothing in the output to indicate it. We hit
   this: a 5-question run left rows behind, and the next 100-question run
   reported 84% while rows 0-4 still carried input_len 25373/20374/16393/6468/
   20727 against 108-146 for the rest. Over the 95 clean rows it was 86.3%; the
   5 stale rows scored 2/5.

   load_existing_results now takes force_rerun and returns empty state when it
   is set. Resume behaviour without the flag is unchanged.

Verified: with a saved five-row file, the default path reloads 5 rows and
resumes at query 5; with force_rerun it reloads 0 and resumes at 0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant