tex.ch: fix 7 integer overflow vulnerabilities in tex.web arithmetic

[dangerzig]

Summary

A systematic review of tex.web (Version 3.141592653) identified 7 integer overflow vulnerabilities under the 32-bit Pascal semantics assumed by the WEB source. Most are currently masked on 64-bit web2c platforms but represent latent incorrectness. All patches use TeX's own overflow-safe primitives.

Bug	Section	Description	Fix
DZ1	S457	Magnification ratio overflow	xn_over_d
DZ2	S762	Delimiter factor overflow	xn_over_d
DZ4	S859	Demerits accumulation overflow	mult_and_add
DZ5	S890	Penalty accumulation overflow	mult_and_add
DZ6	S986, S1008-1010	Page insertion arithmetic overflow (5 sites)	xn_over_d
DZ7	S1005	Page cost overflow	mult_and_add
DZ8	S1236	\advance has no overflow protection	inline check

DZ8 is the only bug demonstrable on current 64-bit TeX Live:

\newcount\big \big=2000000000
\advance\big by 1000000000
\message{\the\big}  % prints -1294967296 (silent wraparound)

After the patch, this correctly reports "Arithmetic overflow".

Note: DZ8 cannot use mult_and_add because mult_and_add's own overflow check (max_answer+y) itself overflows when max_answer = max_int. An inline check is used instead.

Companion issue

The hyphenation exception leak (DZ3) is filed separately as #82.

Trip test impact

Two categories of expected differences in trip.log:

1. DZ6 changes rounding order (divide-then-multiply to multiply-then-divide), producing slightly more accurate page goal and insertion-split values throughout.
2. DZ3 (in tex.ch: fix hyphenation exception slot and memory leak (S941) #82) reduces string pool memory by ~5 words at every Memory usage report.

tripin.log and trip.typ are identical. Full diffs are in the attached trip-test-diffs.txt.

Context

DZ1-DZ3 were reported to Professor Knuth by letter (March 2026) and to tex-k@tug.org. DZ4-DZ8 were found during the same review. The complete changefile patch was sent to the list for review in the thread "Three potential bugs in tex.web (Version 3.141592653)".

[dangerzig]

Attached files

Patch: tex-ch-overflow-patches.diff (unified diff against TeX Live 2025 tex.ch)
Report: tex-ch-overflow-report.pdf (4-page cover document with detailed analysis)
Trip test diffs: trip-test-diffs.txt

These files are too large to paste inline. They have been sent to tex-k@tug.org and are available upon request.

[dangerzig]

tex-ch-overflow-patches.diff

--- tex.ch.orig	2026-04-15 10:30:02
+++ tex.ch	2026-04-28 13:12:46
@@ -1601,6 +1601,39 @@
     if cur_val_level<>mu_val then mu_error;
     goto attach_sign;
     end;
+@z
+
+% Bug DZ1: Integer overflow in magnification ratio (section 457).
+% The subexpression 1000*f+@'200000*remainder can exceed 2^31-1
+% when mag=32768, since 1000*65535+65536*32767 = 2,212,953,112.
+% Add a local variable t to save the remainder, then use xn_over_d.
+@x [26.450] l.8886 - overflow-safe magnification ratio
+@!save_cur_val:integer; {temporary storage of |cur_val|}
+@y
+@!save_cur_val:integer; {temporary storage of |cur_val|}
+@!t:integer; {temporary for overflow-safe magnification}
+@z
+
+@x [26.457] l.9000 - overflow-safe magnification ratio
+@ @<Adjust \(f)for the magnification ratio@>=
+begin prepare_mag;
+if mag<>1000 then
+  begin cur_val:=xn_over_d(cur_val,1000,mag);
+  f:=(1000*f+@'200000*remainder) div mag;
+  cur_val:=cur_val+(f div @'200000); f:=f mod @'200000;
+  end;
+end
+@y
+@ @<Adjust \(f)for the magnification ratio@>=
+begin prepare_mag;
+if mag<>1000 then
+  begin cur_val:=xn_over_d(cur_val,1000,mag);
+  t:=remainder; {save before next |xn_over_d| overwrites}
+  f:=xn_over_d(f,1000,mag); {overflow-safe}
+  f:=f+(@'200000*t+remainder) div mag; {safe: |@'200000*t| $< 2^{31}$}
+  cur_val:=cur_val+(f div @'200000); f:=f mod @'200000;
+  end;
+end
 @z
 
 % See above change "term_input: set limit when fatal_error" for references.
@@ -2766,6 +2799,19 @@
     begin c:=rem_byte(cur_i); i:=char_info(cur_f)(c);
 @y
     begin c:=rem_byte(cur_i); i:=orig_char_info(cur_f)(c);
+@z
+
+% Bug DZ2: Integer overflow in delimiter factor calculation (section 762).
+% Change DX304 (6 Sep 1982) reordered from delta1*delimiter_factor div 500
+% to (delta1 div 500)*delimiter_factor, but overflow still occurs when
+% delimiter_factor > ~1000 and delta1 is near max_dimen.
+% Replace with xn_over_d for overflow-safe computation.
+% Note: this changes rounding (multiply-then-divide vs divide-then-multiply),
+% giving slightly more accurate results. Difference is at most ~2sp.
+@x [36.762] l.15025 - overflow-safe delimiter factor
+delta:=(delta1 div 500)*delimiter_factor;
+@y
+delta:=xn_over_d(delta1,delimiter_factor,500);
 @z
 
 % disabled in original tex-src-special.ch
@@ -2784,6 +2830,40 @@
 if every_cr<>null then begin_token_list(every_cr,every_cr_text);
  @z
 
+% Demerits overflow (section 859).  The d*d and pi*pi computations are
+% carefully guarded, but \doublehyphendemerits, \finalhyphendemerits,
+% and \adjdemerits are unchecked assign_int parameters added with raw +.
+% Setting any to a value near max_int causes overflow.
+% Fix: use mult_and_add(1,...) for overflow-safe addition, clamping
+% to awful_bad since demerits are only used for comparison.
+@x [38.859] l.16906 - overflow-safe demerits accumulation
+if (break_type=hyphenated)and(type(r)=hyphenated) then
+  if cur_p<>null then d:=d+double_hyphen_demerits
+  else d:=d+final_hyphen_demerits;
+if abs(fit_class-fitness(r))>1 then d:=d+adj_demerits;
+@y
+if (break_type=hyphenated)and(type(r)=hyphenated) then
+  if cur_p<>null then d:=mult_and_add(1,d,double_hyphen_demerits,awful_bad)
+  else d:=mult_and_add(1,d,final_hyphen_demerits,awful_bad);
+if abs(fit_class-fitness(r))>1 then d:=mult_and_add(1,d,adj_demerits,awful_bad);
+@z
+
+% Penalty accumulation overflow (section 890, line 17447).
+% \interlinepenalty, \clubpenalty, \widowpenalty, \brokenpenalty are
+% unchecked assign_int parameters accumulated with raw +.
+% Extreme values can overflow.  Clamp to max_dimen.
+@x [39.890] l.17447 - overflow-safe penalty accumulation
+  begin pen:=inter_line_penalty;
+  if cur_line=prev_graf+1 then pen:=pen+club_penalty;
+  if cur_line+2=best_line then pen:=pen+final_widow_penalty;
+  if disc_break then pen:=pen+broken_penalty;
+@y
+  begin pen:=inter_line_penalty;
+  if cur_line=prev_graf+1 then pen:=mult_and_add(1,pen,club_penalty,@'7777777777);
+  if cur_line+2=best_line then pen:=mult_and_add(1,pen,final_widow_penalty,@'7777777777);
+  if disc_break then pen:=mult_and_add(1,pen,broken_penalty,@'7777777777);
+@z
+
 @x [42.920] l.18111 - bigtrie: allow larger hyphenation tries.
 Comparatively few different number sequences $n_0\ldots n_k$ actually occur,
 since most of the |n|'s are generally zero. Therefore the number sequences
@@ -3033,9 +3113,8 @@
 @y
 {repeat hyphenation exception; flushing old data}
 flush_string; s:=hyph_word[h]; {avoid |slow_make_string|!}
+flush_list(hyph_list[h]); {free old hyphen position list}
 decr(hyph_count);
-{ We could also |flush_list(hyph_list[h]);|, but it interferes
-  with \.{trip.log}. }
 goto found;
 not_found:
 @z
@@ -3246,9 +3325,71 @@
 
 @x [43.966] l.18841 - bigtrie: Larger tries.
 @!h:two_halves; {template used to zero out |trie|'s holes}
+@y
+@z
+
+%%%%%%%% Overflow-safe page insertion arithmetic (DZ, March 2026).
+% The page builder uses \count n as a scaling factor for insertions.
+% The code divides first (like DX304), but the subsequent raw
+% multiplication can still overflow when count(n) > ~2000.
+% Replace x_over_n(...,1000)*count(n) with xn_over_d(...,count(n),1000).
+% Note: rounding changes (multiply-then-divide vs divide-then-multiply).
+
+% Diagnostic output (line 19300).
+@x [45.986] l.19299 - overflow-safe insertion scaling
+      if count(t)=1000 then t:=height(r)
+      else t:=x_over_n(height(r),1000)*count(t);
 @y
+      if count(t)=1000 then t:=height(r)
+      else t:=xn_over_d(height(r),count(t),1000);
 @z
 
+% Page cost overflow (section 1005, line 19558).
+% The sum b+pi+insert_penalties is computed before the >=awful_bad
+% guard fires, so extreme values of pi and insert_penalties can
+% overflow the raw + before the check clamps the result.
+% Fix: use mult_and_add for overflow-safe addition, clamping to awful_bad.
+@x [45.1005] l.19558 - overflow-safe page cost
+    else  if b<inf_bad then c:=b+pi+insert_penalties
+@y
+    else  if b<inf_bad then begin c:=mult_and_add(1,b,pi,awful_bad);
+      c:=mult_and_add(1,c,insert_penalties,awful_bad); end
+@z
+
+% First insertion (line 19615).
+@x [45.1008] l.19614 - overflow-safe insertion scaling
+  if count(n)=1000 then h:=height(p)
+  else h:=x_over_n(height(p),1000)*count(n); {this much room is needed}
+@y
+  if count(n)=1000 then h:=height(p)
+  else h:=xn_over_d(height(p),count(n),1000); {this much room is needed}
+@z
+
+% Page insertion node creation (line 19639).
+@x [45.1009] l.19638 - overflow-safe insertion scaling
+if count(n)=1000 then h:=height(r)
+else h:=x_over_n(height(r),1000)*count(n);
+@y
+if count(n)=1000 then h:=height(r)
+else h:=xn_over_d(height(r),count(n),1000);
+@z
+
+% Insertion split: available space (line 19667).
+@x [45.1010] l.19667 - overflow-safe insertion scaling
+  if count(n)<>1000 then w:=x_over_n(w,count(n))*1000;
+@y
+  if count(n)<>1000 then w:=xn_over_d(w,1000,count(n));
+@z
+
+% Insertion split: best_height_plus_depth (line 19674).
+@x [45.1010] l.19673 - overflow-safe insertion scaling
+if count(n)<>1000 then
+  best_height_plus_depth:=x_over_n(best_height_plus_depth,1000)*count(n);
+@y
+if count(n)<>1000 then
+  best_height_plus_depth:=xn_over_d(best_height_plus_depth,count(n),1000);
+@z
+
 %%
 %% We can rewrite the original code after "main_loop_move+2" upto the
 %% "tail_append(lig_stack)" in module 1036 as
@@ -3421,6 +3562,29 @@
 else begin n:=cur_chr; get_r_token; p:=cur_cs; define(p,relax,256);
 @z
 
+% \advance overflow (section 1236, line 23135).
+% \multiply uses mult_integers (overflow-safe, sets arith_error),
+% but \advance uses raw Pascal + with no overflow check.
+% Fix: check for overflow on integer registers before adding.
+% We cannot use mult_and_add with max_answer=@'17777777777
+% because mult_and_add's own overflow check (max_answer+y)
+% would itself overflow. Instead, check directly.
+% Dimension overflow is already caught by scan_normal_dimen.
+@x [49.1236] l.23135 - overflow-safe \advance for integer registers
+  begin if p=int_val then scan_int@+else scan_normal_dimen;
+  if q=advance then cur_val:=cur_val+eqtb[l].int;
+  end
+@y
+  begin if p=int_val then scan_int@+else scan_normal_dimen;
+  if q=advance then
+    if (p=int_val)and@|
+      (((cur_val>0)and(eqtb[l].int>@'17777777777-cur_val))or@|
+      ((cur_val<0)and(eqtb[l].int<(-@'17777777777)-cur_val))) then
+      begin arith_error:=true; cur_val:=0; end
+    else cur_val:=cur_val+eqtb[l].int;
+  end
+@z
+
 @x [49.1252] l.23309 - INI = VIR, so have to do runtime test.
     begin @!init new_patterns; goto done;@;@+tini@/
 @y

trip-test-diffs.txt

Trip test differences with DZ1-DZ8 patches applied
Built from texlive-source (TeX Live 2027/dev) on 2026-04-28

=== tripin.log ===

=== trip.typ ===

=== trip.fot ===
75c75
< Memory usage before: 334&431; after: 292&418; still untouched: 175
---
> Memory usage before: 334&426; after: 292&413; still untouched: 87

=== trip.log ===
128c128
< % t=0.0 plus 3.0filll minus 9.0 g=1470.63013 b=0 p=0 c=-101#
---
> % t=0.0 plus 3.0filll minus 9.0 g=1470.6 b=0 p=0 c=-101#
152c152
< % t=1050.0 plus 44.0 plus 3.0filll minus 19.0 g=1470.63013 b=0 p=0 c=-50
---
> % t=1050.0 plus 44.0 plus 3.0filll minus 19.0 g=1470.6 b=0 p=0 c=-50
176c176
< > 1470.63013pt.
---
> > 1470.6pt.
206c206
< % t=1060.0 plus 127.0 plus 8.0filll minus 27.0 g=1255.43756 b=0 p=0 c=-5
---
> % t=1060.0 plus 127.0 plus 8.0filll minus 27.0 g=1255.40001 b=0 p=0 c=-5
231c231
< > 1255.43756pt.
---
> > 1255.40001pt.
261c261
< % t=1070.0 plus 210.0 plus 8.0filll minus 35.0 g=1055.44061 b=7 p=0 c=-4
---
> % t=1070.0 plus 210.0 plus 8.0filll minus 35.0 g=1055.40001 b=7 p=0 c=-4
286c286
< > 1055.44061pt.
---
> > 1055.40001pt.
315c315
< % split200 to -51.11694,0.0 p=999
---
> % split200 to -51.19998,0.0 p=999
319c319
< % t=1080.0 plus 293.0 plus 8.0filll minus 43.0 g=1055.44061 b=18 p=0 c=9
---
> % t=1080.0 plus 293.0 plus 8.0filll minus 43.0 g=1055.40001 b=18 p=0 c=9
365c365
< % t=1040.0 plus 376.0 plus 8.0filll minus 51.0 g=1055.44061 b=0 p=0 c=94
---
> % t=1040.0 plus 376.0 plus 8.0filll minus 51.0 g=1055.40001 b=0 p=0 c=94
367c367
< % t=1050.0 plus 417.0 plus 8.0filll minus 51.0 g=1055.44061 b=0 p=-5000 
---
> % t=1050.0 plus 417.0 plus 8.0filll minus 51.0 g=1055.40001 b=0 p=-5000 
488,490c488,490
<  goal height 1055.44061
< \insert100 adds 535.36987, #1 might split
< \insert200 adds 405.19714, #3 might split
---
>  goal height 1055.40001
> \insert100 adds 535.4, #1 might split
> \insert200 adds 405.2, #3 might split
506c506
< % t=2061.0 plus 458.0 plus -1.0fil plus 8.0filll minus 51.0 g=1055.44061
---
> % t=2061.0 plus 458.0 plus -1.0fil plus 8.0filll minus 51.0 g=1055.40001
549,552c549,551
< % t=0.0 plus 3.0filll minus 9.0 g=1470.63013 b=0 p=0 c=-101#
< % t=1050.0 plus 44.0 plus 3.0filll minus 19.0 g=1470.63013 b=0 p=0 c=-10
< 1#
< % t=1060.0 plus 127.0 plus 8.0filll minus 27.0 g=1255.43756 b=0 p=0 c=-1
---
> % t=0.0 plus 3.0filll minus 9.0 g=1470.6 b=0 p=0 c=-101#
> % t=1050.0 plus 44.0 plus 3.0filll minus 19.0 g=1470.6 b=0 p=0 c=-101#
> % t=1060.0 plus 127.0 plus 8.0filll minus 27.0 g=1255.40001 b=0 p=0 c=-1
554c553
< % t=1070.0 plus 210.0 plus 8.0filll minus 35.0 g=1055.44061 b=7 p=0 c=-9
---
> % t=1070.0 plus 210.0 plus 8.0filll minus 35.0 g=1055.40001 b=7 p=0 c=-9
556,557c555,556
< % split200 to -51.11694,0.0 p=999
< % t=1080.0 plus 293.0 plus 8.0filll minus 43.0 g=1055.44061 b=18 p=0 c=9
---
> % split200 to -51.19998,0.0 p=999
> % t=1080.0 plus 293.0 plus 8.0filll minus 43.0 g=1055.40001 b=18 p=0 c=9
559c558
< % t=1040.0 plus 376.0 plus 8.0filll minus 51.0 g=1055.44061 b=0 p=0 c=89
---
> % t=1040.0 plus 376.0 plus 8.0filll minus 51.0 g=1055.40001 b=0 p=0 c=89
561c560
< % t=1050.0 plus 417.0 plus 8.0filll minus 51.0 g=1055.44061 b=0 p=-5000 
---
> % t=1050.0 plus 417.0 plus 8.0filll minus 51.0 g=1055.40001 b=0 p=-5000 
563c562
< % t=2061.0 plus 458.0 plus 8.0filll minus 51.0 g=1055.44061 b=* p=-333 c
---
> % t=2061.0 plus 458.0 plus 8.0filll minus 51.0 g=1055.40001 b=* p=-333 c
761c760
< \vbox(1055.44061+0.0)x45.0, glue set 0.01305 []}
---
> \vbox(1055.40001+0.0)x45.0, glue set 0.01295 []}
774c773
< \vbox(1055.44061+0.0)x45.0, glue set 0.01305
---
> \vbox(1055.40001+0.0)x45.0, glue set 0.01295
789,795c788,794
< % t=0.0 plus 8.0filll minus 9.0 g=924.23029 b=0 p=0 c=0#
< % t=7.0 plus 44.0 plus 8.0filll minus 9.0 g=924.23029 b=0 p=0 c=0#
< % t=17.0 plus 127.0 plus 8.0filll minus 17.0 g=924.23029 b=0 p=0 c=0#
< % t=27.0 plus 210.0 plus 8.0filll minus 25.0 g=924.23029 b=0 p=0 c=0#
< % t=-13.0 plus 293.0 plus 8.0filll minus 33.0 g=924.23029 b=0 p=0 c=0#
< % split200 to 1832.4585,10000.0 p=-10000
< % t=1008.0 plus 375.0 plus 8.0filll minus 33.0 g=-4075.76971 b=* p=-333 
---
> % t=0.0 plus 8.0filll minus 9.0 g=924.20001 b=0 p=0 c=0#
> % t=7.0 plus 44.0 plus 8.0filll minus 9.0 g=924.20001 b=0 p=0 c=0#
> % t=17.0 plus 127.0 plus 8.0filll minus 17.0 g=924.20001 b=0 p=0 c=0#
> % t=27.0 plus 210.0 plus 8.0filll minus 25.0 g=924.20001 b=0 p=0 c=0#
> % t=-13.0 plus 293.0 plus 8.0filll minus 33.0 g=924.20001 b=0 p=0 c=0#
> % split200 to 1832.40002,10000.0 p=-10000
> % t=1008.0 plus 375.0 plus 8.0filll minus 33.0 g=-4075.79999 b=* p=-333 
870c869
< \vbox(924.23029+1.0)x0.0, glue set 3.19875 []}
---
> \vbox(924.20001+1.0)x0.0, glue set 3.19864 []}
883c882
< \vbox(924.23029+1.0)x0.0, glue set 3.19875
---
> \vbox(924.20001+1.0)x0.0, glue set 3.19864
889c888
< % split200 to 3955.99365,10000.0 p=-10000
---
> % split200 to 3956.0,10000.0 p=-10000
2190c2189
< Memory usage before: 950&531; after: 546&448; still untouched: 706
---
> Memory usage before: 950&526; after: 546&443; still untouched: 711
2576c2575
< Memory usage before: 339&413; after: 236&405; still untouched: 706
---
> Memory usage before: 339&408; after: 236&400; still untouched: 711
3617c3616
< Memory usage before: 2170&521; after: 1181&459; still untouched: 175
---
> Memory usage before: 2170&516; after: 1181&454; still untouched: 87
3952c3951
< Memory usage before: 640&415; after: 397&410; still untouched: 175
---
> Memory usage before: 640&410; after: 397&405; still untouched: 87
4054c4053
< Memory usage before: 235&401; after: 178&399; still untouched: 175
---
> Memory usage before: 235&396; after: 178&394; still untouched: 87
4507c4506
< Memory usage before: 651&432; after: 417&418; still untouched: 175
---
> Memory usage before: 651&427; after: 417&413; still untouched: 87
4829c4828
< Memory usage before: 776&458; after: 464&452; still untouched: 175
---
> Memory usage before: 776&453; after: 464&447; still untouched: 87
6597c6596
< Memory usage before: 819&428; after: 720&426; still untouched: 175
---
> Memory usage before: 819&423; after: 720&421; still untouched: 87
7157c7156
< Memory usage before: 895&439; after: 290&420; still untouched: 175
---
> Memory usage before: 895&434; after: 290&415; still untouched: 87
7290c7289
< Memory usage before: 334&431; after: 292&418; still untouched: 175
---
> Memory usage before: 334&426; after: 292&413; still untouched: 87
7300c7299
<  2825 words of memory out of 3000
---
>  2913 words of memory out of 3000