Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/benchmark-history.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ env:
*SignificanceBenchmarks.ReduceToThree
*TextBenchmarks.Parse
*ConversionBenchmarks.FromDouble
*AbstractionCostBenchmarks.*
# Short runs: three iterations is enough for a trend line, and a release should not tie up a
# runner for half an hour.
BENCHMARK_JOB: short
Expand Down
162 changes: 162 additions & 0 deletions SignificantNumber.Benchmarks/AbstractionCostBenchmarks.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
// Copyright (c) 2023-2026 ktsu-dev contributors

namespace ktsu.SignificantNumber.Benchmarks;

using System.Globalization;

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;

/// <summary>
/// Measures what this type costs against the same arithmetic on a bare <see cref="double"/>.
/// </summary>
/// <remarks>
/// <para>
/// Every other class here answers "how long does this operation take", which is only readable next
/// to something. This one supplies the something: the primitive a caller would otherwise have
/// used. The same class, with the same loops and the same methodology, is in ktsu.PreciseNumber and
/// ktsu.Semantics, so the three answers are comparable with each other as well as with
/// <see cref="double"/>. Against ktsu.PreciseNumber in particular the difference is what
/// significance tracking adds, since this type is built on that one.
/// </para>
/// <para>
/// The bare method is the BenchmarkDotNet baseline, so the answer is the <c>Ratio</c> column rather
/// than two rows divided by hand. A ratio here is not expected to be 1.00 and is not a defect when
/// it is not: every operation pays twice, once for the arbitrary-precision arithmetic and again for
/// rounding the result back to the significance the operands justify, and both are costs paid for
/// something a <see cref="double"/> cannot do at all. What the number is for is watching that cost
/// across releases.
/// </para>
/// <para>
/// <b>Why these are loops.</b> A single operation over operands that do not change is
/// loop-invariant, and the JIT hoists it out of the measurement entirely — for
/// <see cref="double"/> that leaves a method indistinguishable from an empty one, and a ratio
/// against an empty method means nothing. Here each iteration feeds the next, so there is nothing
/// to hoist and both sides are measurable.
/// </para>
/// <para>
/// <b>Which way the loop biases the answer.</b> Both sides pay the same counter and branch, and it
/// is a dependency chain, so most of that overlaps the arithmetic; whatever does not is added
/// equally to numerator and denominator and pulls the ratio toward 1.00. A ratio here is therefore
/// a floor on the real cost rather than the whole of it.
/// </para>
/// <para>
/// <b>Why the operands stay bounded, and why they are short.</b> The number underneath carries as
/// many digits as the arithmetic produces, so a chain that compounded its operand would measure
/// that growth rather than the operation; both loops accumulate instead. The operands are also
/// chosen to be values a <see cref="double"/> can hold, so the two sides are doing the same
/// arithmetic on the same numbers rather than being handed different problems. How the cost grows
/// with digits is a different question, and <see cref="ArithmeticBenchmarks"/> answers it across
/// its <c>Digits</c> axis.
/// </para>
/// </remarks>
[MemoryDiagnoser]
[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]
[CategoriesColumn]
public class AbstractionCostBenchmarks
{
/// <summary>
/// Operations per invocation. Enough that the loop's own cost is a small share of the work,
/// few enough that the arbitrary-precision side still finishes an iteration promptly.
/// </summary>
private const int Operations = 256;

private const string SeedText = "1234.5678901234";
private const string StepText = "0.0009765625";
private const string OtherText = "3.14159265358979";

private double bareSeed;
private double bareStep;
private double bareOther;

// Assigned in GlobalSetup before anything is measured. Initialised here because this type
// was a class before 2.0, where an unassigned field is a null reference the compiler
// rejects; from 2.0 it is a struct and this is simply its default. The backfill measures
// those releases too, so the file has to compile against both shapes.
private SignificantNumber significantSeed = default!;
private SignificantNumber significantStep = default!;
private SignificantNumber significantOther = default!;

/// <summary>
/// Prepares the operands, parsed from the same text on both sides.
/// </summary>
[GlobalSetup]
public void Setup()
{
bareSeed = double.Parse(SeedText, CultureInfo.InvariantCulture);
bareStep = double.Parse(StepText, CultureInfo.InvariantCulture);
bareOther = double.Parse(OtherText, CultureInfo.InvariantCulture);

significantSeed = SignificantNumber.Parse(SeedText, CultureInfo.InvariantCulture);
significantStep = SignificantNumber.Parse(StepText, CultureInfo.InvariantCulture);
significantOther = SignificantNumber.Parse(OtherText, CultureInfo.InvariantCulture);
}

/// <summary>Adds along a chain, on a bare double.</summary>
/// <returns>The accumulated value.</returns>
[BenchmarkCategory("Add")]
[Benchmark(Baseline = true, OperationsPerInvoke = Operations)]
public double BareAdd()
{
double accumulator = bareSeed;

for (int i = 0; i < Operations; i++)
{
accumulator += bareStep;
}

return accumulator;
}

/// <summary>Adds along the same chain, on this type.</summary>
/// <returns>The accumulated value.</returns>
[BenchmarkCategory("Add")]
[Benchmark(OperationsPerInvoke = Operations)]
public SignificantNumber SignificantAdd()
{
SignificantNumber accumulator = significantSeed;

for (int i = 0; i < Operations; i++)
{
accumulator += significantStep;
}

return accumulator;
}

/// <summary>Multiplies and accumulates, on a bare double.</summary>
/// <returns>The accumulated value.</returns>
[BenchmarkCategory("Multiply")]
[Benchmark(Baseline = true, OperationsPerInvoke = Operations)]
public double BareMultiply()
{
double accumulator = 0d;
double value = bareSeed;

for (int i = 0; i < Operations; i++)
{
accumulator += value * bareOther;
value += bareStep;
}

return accumulator;
}

/// <summary>Multiplies and accumulates over the same values, on this type.</summary>
/// <returns>The accumulated value.</returns>
[BenchmarkCategory("Multiply")]
[Benchmark(OperationsPerInvoke = Operations)]
public SignificantNumber SignificantMultiply()
{
SignificantNumber accumulator = SignificantNumber.Zero;
SignificantNumber value = significantSeed;

for (int i = 0; i < Operations; i++)
{
accumulator += value * significantOther;
value += significantStep;
}

return accumulator;
}
}
38 changes: 38 additions & 0 deletions SignificantNumber.Benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,44 @@ rather than wrapping it, so `ToString` was an inherited member: measuring it wou
that package here, and the reference makes `Parse` and `GetHashCode` ambiguous against their
inherited counterparts. One operation is not worth the whole release history before 2.0.

## What this type costs against a bare double

`AbstractionCostBenchmarks` is the one benchmark here whose answer is a ratio rather than a
duration. Every other class says how long an operation takes, which is only readable beside
something; this supplies the something — the primitive a caller would otherwise have used.

The same class, with the same loops and the same methodology, is in `ktsu.PreciseNumber` and
`ktsu.Semantics`. Against PreciseNumber in particular the difference is what significance tracking
adds, since this type is built on that one.

| release | `Add` | `Multiply` |
|---|---|---|
| 1.3.0 | 1377.7× | 5809.6× |
| 1.4.40 | 1352.4× | 5944.6× |
| 2.0.0 | **89.7×** | **550.5×** |
| 2.0.1 | 89.9× | 569.3× |

Becoming a value type in 2.0 took roughly **15× off add and 10× off multiply**. For comparison,
the same change in `ktsu.PreciseNumber` underneath moved its ratios by about 15% — so most of what
2.0 recovered here was this layer's own allocation, not the number beneath it.

**The ratio is not expected to be 1 and is not a defect for being large.** Every operation pays
twice: once for the arbitrary-precision arithmetic and again for rounding the result back to the
significance the operands justify, and both buy something a `double` cannot do at all. What the
chart's third section is for is noticing the day it moves.

Three things decide how the number should be read:

- **These are loops.** A single operation over operands that do not change is loop-invariant and
the JIT hoists it out, which would leave the `double` side indistinguishable from an empty method
and the ratio meaningless. Each iteration feeds the next, so there is nothing to hoist.
- **The loop's own cost biases toward 1**, being paid by both sides, so a ratio is a floor on the
real cost rather than the whole of it.
- **Both loops accumulate rather than compound**, because the number underneath carries as many
digits as the arithmetic produces and a compounding chain would measure that growth instead of
the operation. How the cost grows with digits is a different question, and `ArithmeticBenchmarks`
answers it across the `Digits` axis.

## Reading the results

Most classes are parameterised by `Digits` (8, 30, 200). That axis is the point: significance is
Expand Down
Loading