fix(streams): stop a bad byte from stalling transcript ingestion

[posthog]

Problem

• A single bad byte in one agent transcript permanently stops the daemon from ingesting the rest of that session, and no one is told.
• read_jsonl_line read each line with BufRead::read_line into a String, so any byte that is not valid UTF-8 came back as an InvalidData io error. All eight agent readers under src/streams/agents/ wrap that as StreamError::Transient, which the stream worker retries four times (5s, 30s, 5m, 30m) and then drops.
• The failing condition is a property of the bytes at a fixed offset, not a temporary fault, so no retry can ever clear it. The watermark advances only after a successful batch, so the stream stalls at that offset forever and every later event in the session transcript is lost silently.

Changes

• Read the line as raw bytes with read_until(b'\n') and decode with String::from_utf8_lossy instead of read_line.
• A complete line that is not valid UTF-8 now decodes lossily and advances the offset. If the decoded text is not valid JSON, the reader skips it through the existing malformed-JSON path, so the offset still moves forward.
• A line without a trailing newline stays Partial, which keeps the concurrent-writer behavior and also covers a half-written multi-byte character at the tail of a growing file.
• The function signature does not change, so all eight readers keep working with no edits.

Tests

• Added unit tests: a complete line with an invalid byte reports Complete with the raw byte count and decodes to a replacement character; a bad line does not block the next line; a half-written multi-byte character with no newline stays Partial.

Note

This sandbox has no Rust toolchain, so task test, task lint, and task fmt could not run locally. CI exercises them on this PR.

Agent context

• The fix keeps the &mut String signature to stay minimal; the alternative (a &mut Vec<u8> buffer to reuse the allocation) would touch all eight callers for a negligible per-line cost next to JSON parsing.

---

Created with PostHog Desktop from this inbox report.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}