fix: stop TUI truncation helpers from splitting multi-byte UTF-8 runes - #1160
Open
STiFLeR7 wants to merge 1 commit into
Open
fix: stop TUI truncation helpers from splitting multi-byte UTF-8 runes#1160STiFLeR7 wants to merge 1 commit into
STiFLeR7 wants to merge 1 commit into
Conversation
truncStr, firstN, and lastN (internal/render/helpers.go), ptrunc
(proxy.go), and truncateShellLine (terminal.go) all sliced strings on
raw byte offsets. When a cut landed mid-rune, the result was invalid
UTF-8 and the terminal rendered mojibake; the same byte-vs-column
mismatch also cut CJK/emoji lines short, since those runes spend 2-4
bytes but only 1-2 display columns of the budget.
Switch all five to github.com/charmbracelet/x/ansi, already a direct
dependency and already the established pattern for exactly this in
internal/app (view.go). Truncate/TruncateLeft are grapheme-aware and
operate on display columns, not bytes, so a cut can no longer split a
rune and CJK/emoji now correctly consume 2 columns instead of 1.
- truncStr/firstN ("keep the start"): ansi.Truncate(s, n, "")
- lastN ("keep the end"): reverse via ansi.TruncateLeft(s, w-n, "")
where w is the string's total display width
- ptrunc/truncateShellLine (truncate-with-ellipsis): ansi.Truncate
already reserves the tail's width internally, so this is a direct
swap of the same max/length budget, no arithmetic needed
Added helpers_test.go with the issue's own repro (asserting valid
UTF-8 out of all five paths) plus per-function behavior tests.
Fixes usestrix#1152.
Contributor
Greptile SummaryThe PR replaces byte-offset TUI truncation with grapheme-aware, display-column-based ANSI helpers to prevent invalid UTF-8 and mojibake.
Confidence Score: 5/5The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issues identified. The changed helpers consistently use display-column-aware truncation, preserve valid UTF-8, retain existing caller budgets, and are covered by targeted regression tests. Important Files Changed
Reviews (1): Last reviewed commit: "fix: stop TUI truncation helpers from sp..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
truncStr,firstN,lastN(internal/render/helpers.go),ptrunc(proxy.go), andtruncateShellLine(terminal.go) all slice on raw byte offsets. When a cut lands mid-rune, the output is invalid UTF-8 and the terminal shows�. Same root cause cuts CJK/emoji lines short, since those runes spend 2-4 bytes but only 1-2 display columns of the truncation budget.Fix
Switch all five to
github.com/charmbracelet/x/ansi(Truncate/TruncateLeft), already a direct dependency and already the established pattern for this exact problem ininternal/app(view.go). Both are grapheme-aware and count display columns, not bytes.truncStr/firstN("keep the start"):ansi.Truncate(s, n, "")lastN("keep the end"): reversed viaansi.TruncateLeft(s, w-n, "")wherewis the string's total display width —ansi.TruncateLeftremoves N columns from the start, so keeping the lastncolumns means removingw - n.ptrunc/truncateShellLine(truncate-with-ellipsis):ansi.Truncatealready reserves the tail string's width internally, so this is a direct swap of the samemax/maxLineLengthbudget — no manual-3arithmetic needed anymore.Fixes #1152.
Testing
helpers_test.gowith the issue's own repro (TestTruncSplitsRunes, asserting valid UTF-8 out of all five paths) plus per-function behavior tests (TestFirstNTruncStrKeepStart,TestLastNKeepsEnd,TestPtruncAppendsEllipsisWithinBudget).firstN("中文字ABC", 4)→"中\xe6", invalid UTF-8); restored the fix, confirmed valid output.go build ./...,go vet ./..., and the full existinginternal/rendertest suite all pass with the fix applied.gofmt -lreports no formatting issues.