Skip to content

ZeroTable.GetTable (and WideTable.GetTable) are not thread-safe — broken double-checked locking causes NullReferenceException #11

Description

@tig

Problem

Exception

System.NullReferenceException: Object reference not set to an instance of an object.
  at System.Collections.Generic.Dictionary`2.FindValue(TKey key)
  at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
  at Wcwidth.ZeroTable.GetTable(Unicode version)            ← line 20 (outer TryGetValue, outside lock)
  at Wcwidth.UnicodeCalculator.GetWidth(Int32 value, ...)
  at Terminal.Gui.Text.RuneExtensions.GetColumns(Rune rune)
  at Terminal.Gui.Text.StringExtensions.GetColumns(String str, ...)
  at Terminal.Gui.Text.TextFormatter.FormatAndGetSize(...)
  at Terminal.Gui.ViewBase.DimAuto.Calculate(...)
  at Terminal.Gui.ViewBase.View.SetRelativeLayout(...)

Crash occurs during ApplicationImpl.BeginRaiseIsRunningChangedEvent
EndInitLayoutGetColumns, while the input thread started by
MainLoopCoordinator.StartInputTaskAsync is also initializing concurrently.

Root Cause

Wcwidth 4.0.1 — ZeroTable.GetTable (and WideTable.GetTable) lazily
populate a private static readonly Dictionary<Unicode, int[,]> _lookup using
a broken double-checked locking pattern:

// ZeroTable.cs — same pattern in WideTable.cs
public static int[,] GetTable(Unicode version)
{
    // ⚠️ OUTSIDE the lock — races with the write below
    if (!_lookup.TryGetValue(version, out int[,] value))
    {
        lock (_lock)
        {
            if (_lookup.TryGetValue(version, out value))
                return value;

            value = GenerateTable(version);
            _lookup[version] = value; // ← triggers internal resize under lock
        }
    }
    return value;
}

Dictionary<TKey, TValue> is safe for concurrent reads only. When
_lookup[version] = value (inside the lock on one thread) triggers an internal
resize, the _buckets and _entries arrays are momentarily in an inconsistent
state. A second thread doing the outer TryGetValue (outside the lock)
traverses this inconsistent state, permanently corrupting the static dictionary.

Evidence of Corruption (from debugger locals)

Variable Expected Actual
_lookup.Count 0 or 1 1 (entry was added)
hashCode (uint)key.GetHashCode() 87
i (bucket index) 0–3 for a 1-element dict 773,305,808

The out-of-range bucket index proves the internal hash table is corrupted.
The corruption is permanent for the lifetime of the process.

Concurrency Scenario

Main thread (29084)                    Input thread (17872)
─────────────────────────────────────  ──────────────────────────────────────
ApplicationImpl.Init()
  └─ MainLoopCoordinator
       └─ Task.Run(() => RunInput())  ──► BuildDriverIfPossible()
                                            └─ ... → GetColumns()
                                                  └─ ZeroTable.GetTable()
                                                       outer TryGetValue()  ← reads
  └─ Begin() → EndInit()
       └─ Layout → GetColumns()
             └─ ZeroTable.GetTable()
                  lock acquired
                  _lookup[ver] = val  ← writes (resize)   💥 DATA RACE

Fix

Workaround (applied — Terminal.Gui\App\ApplicationImpl.Lifecycle.cs)

Pre-warm the Wcwidth static cache before the input thread starts, during
single-threaded Init(). This guarantees _lookup is already fully populated
when any background thread could call GetColumns:

// ApplicationImpl.Init(), before CreateDriver() / before input thread starts:
// Pre-warm the Wcwidth static cache to prevent ZeroTable._lookup race condition
_ = UnicodeCalculator.GetWidth (new Rune ('A'));

Calling GetWidth for any rune forces ZeroTable.GetTable(Version_16_0_0) and
WideTable.GetTable(Version_16_0_0) to run on the main thread, populating both
_lookup dictionaries. All subsequent calls from any thread find the key and
return immediately without ever taking the lock or writing, making reads safe.

Location: Terminal.Gui\App\ApplicationImpl.Lifecycle.cs, inside Init(),
immediately before CreateDriver(_driverName).

Upstream Fix (to be done in spectreconsole/wcwidth)

Replace the non-thread-safe Dictionary with ConcurrentDictionary in both
ZeroTable and WideTable:

// Before
private static readonly Dictionary<Unicode, int[,]> _lookup;
private static readonly object _lock;

static ZeroTable()
{
    _lookup = new Dictionary<Unicode, int[,]>();
    _lock = new object();
}

public static int[,] GetTable(Unicode version)
{
    if (!_lookup.TryGetValue(version, out int[,] value))
    {
        lock (_lock)
        {
            if (_lookup.TryGetValue(version, out value))
                return value;
            value = GenerateTable(version);
            _lookup[version] = value;
        }
    }
    return value;
}

// After
private static readonly ConcurrentDictionary<Unicode, int[,]> _lookup = new ();

public static int[,] GetTable(Unicode version)
{
    return _lookup.GetOrAdd(version, static v => GenerateTable(v));
}

This eliminates the lock entirely. ConcurrentDictionary.GetOrAdd is safe for
concurrent callers. Note: GenerateTable may be called more than once under
very high contention (it is not guaranteed to run exactly once), but since it is
a pure, deterministic function with no side effects this is acceptable.


Files Changed

File Change
Terminal.Gui\App\ApplicationImpl.Lifecycle.cs Added UnicodeCalculator.GetWidth(new Rune('A')) pre-warm call in Init()

Upstream Issue

Filed at: https://github.com/spectreconsole/wcwidth/issues (pending)

Title: ZeroTable.GetTable and WideTable.GetTable are not thread-safe —
broken double-checked locking causes NullReferenceException


Cleanup

Once a fixed Wcwidth version is released and Directory.Packages.props is
updated to consume it, remove the pre-warm line from ApplicationImpl.Init().

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions