Skip to content

Fix underline and half-bright being discarded where plain char is unsigned - #1

Open
kay-ws wants to merge 1 commit into
antmicro:masterfrom
kay-ws:fix-underline-unsigned-char
Open

Fix underline and half-bright being discarded where plain char is unsigned#1
kay-ws wants to merge 1 commit into
antmicro:masterfrom
kay-ws:fix-underline-unsigned-char

Conversation

@kay-ws

@kay-ws kay-ws commented Aug 28, 2026

Copy link
Copy Markdown

SGR 4 (underline) and SGR 2 (faint) are silently discarded on targets where
plain char is unsigned: the terminal takes the "represent the decoration as a
color instead" path unconditionally, even though no such color was ever
requested. The substituted index is 255 — rgb(238, 238, 238) in
defaultPalette, against a normal foreground of index 7, rgb(170, 170, 170)
— so the text merely comes out slightly brighter and the failure is easy to
overlook.

What the guards are for

The VGA text console cannot draw an underline, so the Linux console represents
underlined cells as a color change instead. console_codes(4), under Linux
Console Private CSI Sequences
:

ESC [ 1 ; n ] Set color n as the underline color.
ESC [ 2 ; n ] Set color n as the dim color.

VTerm::linux_specific() implements both, and normal_char_attr() applies the
substitution: where a color has been requested, the decoration bit is cleared
and the foreground is replaced. -1 means no substitution requested — draw a
real underline, which fbterm can do because it is not limited to VGA text mode.

Why the sentinel does not survive

src/lib/vterm.h stores that -1 in s8, which src/lib/type.h defines as
plain char, whose signedness is implementation-defined. GCC's documentation
puts it this way:

Each target supported by GCC has a default for what char should be, which
is typically specified in the processor-specific ABI document for that target.

For Arm that document is the AAPCS, whose §8.1.1 Arithmetic Types maps
char to unsigned byte, with the note "LDRB is unsigned".

VTerm::reset() sets both fields to -1, which on such a target reads back as
255, so the guards in VTerm::normal_char_attr() are true unconditionally:

if (a.underline && cur_underline_color != -1) {   // 255 != -1
    a.underline = false;                          // decoration discarded
    a.fcolor = cur_underline_color;               // fg forced to 255
}

if (a.intensity == 0 && cur_halfbright_color != -1) {  // same defect
    a.intensity = 1;
    a.fcolor = cur_halfbright_color;
}

The guards themselves are older than the decoration rendering added in
ccf83cf; before that commit fbterm never drew underlines, so the broken
comparison had no observable effect.

Confirmed on the target with a two-line program:

$ cat sc.c
#include <stdio.h>
int main(){ char c = -1; printf("%s\n", c < 0 ? "SIGNED" : "UNSIGNED"); return 0; }
$ gcc -o sc sc.c && ./sc
UNSIGNED
$ gcc -dM -E -x c /dev/null | grep CHAR_UNSIGNED
#define __CHAR_UNSIGNED__ 1

About the fix

Changing the typedef to signed char is not an option: s8 is also the string
type used throughout the tree, and signed char * is not assignable from a
string literal in C++. -fsigned-char would work and would remove the whole
class of latent s8 sign bugs, but it also flips the interpretation of every
s8 buffer holding UTF-8 bytes, so this patch widens the two fields instead —
they only ever hold -1 to 7. If you would rather take the compiler flag,
that decision belongs to you and this commit can be dropped.

Reproducing

printf '\e[4mUNDERLINE\e[0m  \e[2mFAINT\e[0m  \e[9mSTRIKE\e[0m\n'

Before, on an unsigned-char target: UNDERLINE has no underline and FAINT
is brighter than surrounding text rather than dimmer; STRIKE renders
correctly, since strikethrough has no such substitution. After, all three
render as expected.

That contrast doubles as a discriminator when debugging this area —
strikethrough and underline are emitted by the same fillRect() call in
drawGlyph() and differ only in the y offset, so "strikethrough renders but
underline does not" points at the attribute rather than the geometry.

Verification

  • Built and tested on armv7l (Debian 12) from this branch: before the change no
    underline is drawn at all; after it, ASCII and double-width cells, colored
    underlines and SGR 2 all render correctly.
  • The same binary sources built on x86_64, where char is signed, draw the
    underline correctly with and without this commit — which is what the analysis
    above predicts.
  • SGR 21 (double underline) remains unimplemented; this patch does not
    change that.

I have a shell script covering the underline paths and a larger one covering
the 256-color and direct-color output, if a tests/ entry would be useful.

…nsigned

s8 is a typedef for plain char, whose signedness is implementation-defined.
The Arm procedure call standard maps char to "unsigned byte" (AAPCS32 section
8.1.1, "LDRB is unsigned"), so the -1 sentinel stored in cur_underline_color
and cur_halfbright_color reads back as 255 there and the guards in
normal_char_attr() are true unconditionally:

    if (a.underline && cur_underline_color != -1) {   // 255 != -1
        a.underline = false;
        a.fcolor = cur_underline_color;               // fg forced to 255
    }

Underline (SGR 4) is therefore dropped and the cell is drawn with color index
255, a near-white gray on the default palette, so the failure is easy to
overlook: the text looks almost normal, it just loses the decoration. The
half-bright branch below it has the same defect and replaces the intended
gray with color 255.

The guards predate the decoration rendering added in ccf83cf; before that
commit fbterm never drew underlines, so the bug had no observable effect.

The typedef itself cannot be changed to signed char, because s8 is also the
string type used throughout the tree and signed char * is not assignable from
a string literal in C++. -fsigned-char would work too, but it also changes the
interpretation of every s8 buffer holding UTF-8 bytes, so widening the two
fields seemed the smaller change. They only ever hold -1 to 7.

Verified on armv7l (Debian 12).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant