System.Decimal (29-38 significant digits) support

[kliszaq]

Title

DECIMAL values wider than System.Decimal (29-38 significant digits) cannot be read at all — no lossless accessor, although the full value is already computed internally

Describe the bug

DuckDB supports DECIMAL up to width 38, backed by a 128-bit integer. The ADO.NET reader, however,
funnels every read of a DECIMAL column through System.Decimal, which holds only 28-29
significant digits. For any value wider than that, every public accessor throws — there is no way to
get the value out of the reader, even though the full-precision BigInteger is materialized
internally right before the conversion throws.

To Reproduce

using DuckDB.NET.Data;
using System.Numerics;
 
using var connection = new DuckDBConnection("DataSource=:memory:");
connection.Open();
 
using var command = connection.CreateCommand();
command.CommandText = "SELECT CAST('99999999999999999999999999999999999999' AS DECIMAL(38,0))";
 
using var reader = command.ExecuteReader();
reader.Read();
 
reader.GetValue(0);                      // OverflowException
reader.GetDecimal(0);                    // OverflowException (expected - the value cannot fit)
reader.GetFieldValue<BigInteger>(0);     // no escape hatch: for wide values OverflowException,
                                         // for in-range values InvalidCastException
                                         // ("Unable to cast object of type 'System.Decimal'
                                         //   to type 'System.Numerics.BigInteger'")
reader.GetFieldValue<string>(0);         // same - InvalidCastException for in-range values,
                                         // OverflowException for wide ones

Expected behavior

Some lossless read path for wide DECIMAL values, in any convenient shape, for example:

• GetFieldValue<BigInteger>() returning the unscaled value (scale is available from the schema), or
• GetFieldValue<string>() returning the textual value, or
• a public accessor returning DuckDBDecimal (the struct already exists in DuckDB.NET.Native with
  Width, Scale and a DuckDBHugeInt value).

Actual behavior

System.OverflowException : Value was either too large or too small for a Decimal.
   at System.Numerics.BigInteger.op_Explicit(BigInteger value)
   at DuckDB.NET.Data.DataChunk.Reader.DecimalVectorDataReader.GetDecimal(UInt64 offset)
   at DuckDB.NET.Data.DataChunk.Reader.DecimalVectorDataReader.GetValue(UInt64 offset, Type targetType)
   at DuckDB.NET.Data.DataChunk.Reader.VectorDataReaderBase.GetValue(UInt64 offset)
   at DuckDB.NET.Data.DuckDBDataReader.GetValue(Int32 ordinal)

Where it happens in the code

DuckDB.NET.Data/DataChunk/Reader/DecimalVectorDataReader.cs:

1. GetValue(ulong offset, Type targetType) ignores targetType for DECIMAL columns and always
   calls GetDecimal(offset) — so GetFieldValue<BigInteger>() / GetFieldValue<string>() never
   reach the vector reader and cannot serve as an escape hatch:

   internal override object GetValue(ulong offset, Type targetType)
   {
       if (DuckDBType != DuckDBType.Decimal)
       {
           return base.GetValue(offset, targetType);
       }
   
       return GetDecimal(offset);
   }

2. In the DuckDBType.HugeInt branch of GetDecimal, the full-precision value is already in hand as
   a BigInteger, and is lost one line later in the cast that throws:

   var hugeInt = numericVectorDataReader.GetBigInteger(offset, false);
   var result = (decimal)BigInteger.DivRem(hugeInt, DecimalExtensions.BigIntPowersOfTen[Scale], out var remainder);

3. The bindings do expose a public DuckDBDecimal struct (Width, Scale, DuckDBHugeInt Value),
   but the only API returning it is the legacy row-based
   NativeMethods.Types.DuckDBValueDecimal(ref DuckDBResult, col, row), which is not reachable from
   DuckDBDataReader (the data-chunk path).

Environment

• DuckDB.NET.Data.Full 1.5.5 (latest; the develop branch behaves the same — verified in source)
• .NET 10.0, Windows 10 x64

[Giorgi]

As a workaround, you can cast it to varchar and read it like that