[Release 0.70.0] ~85% NumPy 2.x API coverage: 100+ new np.* APIs, new nuget packages (OpenBLAS & pythonnet), 3 living dashboards

.claude/commands/np-function.md

@@ -28,7 +28,7 @@ NumPy is the source of truth and if NumPy does A, we do A but in NumSharp's C# w
     - Do not implement struct kernel.
 ## 2. Implement np method/s
 - Implement np methods to the fullest, integrating into our existing infrastructure and patterns.
-- Our implementation might differ from NumPy's because NumPy uses C++ macros while we generate IL methods during runtime to achieve peak performance and cpu acceleration. But any input given to NumPy will produce same output with complete parity.
+- Our implementation might differ from NumPy's because NumPy uses C++ macros while we generate IL methods during runtime to achieve peak performance and cpu acceleration on 1k, 100k, 1m/10m length (1m on large dtypes). But any input given to NumPy will produce same output with complete parity.
 - Our implementation must provide same parameters as the NumPy function and support all dtypes NumSharp currently supports.
 - Do not create a function per dtype/NPTypeCode or if-else/switch-case per dtype/NPTypeCode to call a specialized path.
 - Do not use struct kernel pattern.
@@ -41,6 +41,8 @@ NumPy is the source of truth and if NumPy does A, we do A but in NumSharp's C# w
 These cli functions allow rapid development and experimentation.
 Specifying '#:project' and other '#' with paths must be absolute path.
 
+> **Large or quote-dense scripts:** do NOT inline them in a `<<'EOF'` heredoc — the Bash tool wraps every command in `eval '…'` and re-escapes each `'`, so a quote-dense heredoc stops balancing at scale and fails with `unexpected EOF while looking for matching '`. Write the script to a file in the session scratchpad directory (the path given in the system prompt — never the repo cwd, never `/tmp`; a bare `probe.cs` lands in the repo tree) and redirect it in: `dotnet run -c Release - < <scratchpad>/probe.cs` or `python <scratchpad>/probe.py`. Short snippets can stay inline. (Timing/benchmark scripts are file-based anyway — the `benchmark` skill mandates `dotnet run -c Release - < script.cs` with a fresh filename per rebuild.)
+
 ### Benchmarking
 Use "dotnet run <<'EOFDOTNET'" and "python <<'EOFPYTHON'" to produce professional benchmarks.
 

.claude/skills/benchmark/SKILL.md

@@ -2,13 +2,14 @@
 name: benchmark
 description: >-
   NumSharp's NumPy-vs-NumSharp performance harness — the op/dtype/N matrix (C# BenchmarkDotNet vs a
-  warm NumPy process) plus five appended subsystems (nditer, layout, operand, cast, fusion), all in
+  warm NumPy process), unified Managed/OpenBLAS backend profiles, and five appended subsystems
+  (nditer, layout, operand, cast, fusion), all in
   the NPY/NS convention. Use this whenever you add a benchmark for an np.* op, wire a C# benchmark to
   its NumPy twin, run the official suite or a subset, read/interpret the ratio matrix or history
   snapshots, add a whole subsystem, or debug a suspicious measurement (the Debug-taint 2x pitfall,
-  the InProcessEmit toolchain). Trigger on: "benchmark", "add a benchmark", "how fast is <op> vs
+  the InProcessEmit toolchain). Trigger on: "benchmark", "add a benchmark", "how fast is np.add vs
   numpy", "run_benchmark.py", "BenchmarkDotNet", "NPY/NS ratio", "perf comparison", "benchmark
-  <op>", "benchmark-report", "history snapshot", "why is my timing 2x slow". Reach for it before
+  np.add", "benchmark-report", "history snapshot", "why is my timing 2x slow". Reach for it before
   quoting any NumSharp-vs-NumPy speed number.
 ---
 
@@ -20,9 +21,18 @@ is the distilled map + the actionable playbooks. Read `benchmark/CLAUDE.md` when
 ## THE convention: NPY/NS (memorize this)
 
 > **ratio = NumPy_ms / NumSharp_ms.** `>1` = NumSharp **faster**, `<1` = slower, `=1` = parity. **Higher is better.**
-
-Used everywhere — matrices, geomeans, commit messages, every `*_sheet.py`. Icons: ✅ `≥1.0` · 🟡 `≥0.5` · 🟠 `≥0.2`
-· 🔴 `<0.2`. (The legacy `run-benchmarks.ps1` prints the INVERSE NS/NPY — prefer NPY/NS for anything new.)
+> **Timing basis: best window (min)** — ratios compare each side's per-case *minimum*, never the mean. Interference
+> (GC pauses, page-fault storms, ambient machine load) only ever *adds* time, and it lands almost entirely in the
+> NumSharp side's right tail, so comparing means turned machine state into fake ratio regressions. Min-based
+> harnesses run each case to a ~200 ms time budget (a >20 ms/call op runs exactly 100 times); the C# op-matrix keeps
+> BenchmarkDotNet's 50 iterations. Per-case means stay in the JSON (`numpy_mean_ms`/`numsharp_mean_ms`) as tail
+> diagnostics — a large mean/min gap on one side flags a contaminated window.
+
+Used everywhere — matrices, geomeans, commit messages, every `*_sheet.py`. The report's status icons
+(✅/🟡/🟠/🔴 faster→slower · ▫ negligible/excluded · ⚪ pending · ❌ crashed) and their exact cutoffs live in
+`references/run-and-report.md`; the source of truth is `get_status()` + `classify()` in `scripts/merge-results.py`,
+so read those rather than trusting a hardcoded threshold. (The legacy `run-benchmarks.ps1` prints the INVERSE
+NS/NPY — prefer NPY/NS for anything new.)
 
 ## THE pitfall: Debug taints timings ~2×
 
@@ -36,23 +46,45 @@ script assembly, not Core. The BenchmarkDotNet projects are exempt (they mandate
 
 | Side | Where | What |
 |------|-------|------|
-| C# | `benchmark/NumSharp.Benchmark.CSharp/Benchmarks/<Category>/*.cs` | BenchmarkDotNet classes; `[Benchmark(Description="np.foo(a)")]` methods. |
+| C# | `benchmark/NumSharp.Benchmark.CSharp/Benchmarks/<Category>/*.cs` | Core-only BenchmarkDotNet project plus shared benchmark classes; `[Benchmark(Description="np.foo(a)")]` methods. |
+| OpenBLAS C# | `benchmark/NumSharp.Benchmark.CSharp.OpenBLAS/` | Enables one-thread OpenBLAS and reruns the shared official LinearAlgebra classes/config. |
 | NumPy | `benchmark/NumSharp.Benchmark.Python/numpy_benchmark.py` | `run_<suite>_benchmarks(...)` emitting `BenchmarkResult` rows. |
-| Merge | `benchmark/scripts/merge-results.py` | Joins on `(normalize_op_name(name), dtype, N)`. |
+| Merge | `benchmark/scripts/{merge-results,merge-backend-profiles}.py` | Joins language timings, then backend profiles on the exact cell. |
 | Orchestrator | `benchmark/run_benchmark.py` | Builds C#, runs each suite (BDN) + warm NumPy across 1K/100K/10M, merges, snapshots. |
 
+`run_benchmark.py --depth pass|light|measure` is the execution-depth contract shared by BDN and
+NumPy (defined once in `scripts/benchmark_modes.py`, so the two languages can't drift). `pass` is an
+execution gate (one workload call, zero warmups); `light` uses 8 BDN measurements after 3 warmups plus
+one-sixth of NumPy's normal time budget; `measure` is the full 50-measurement / 5-warmup BDN + full
+NumPy publication profile. Pass/light write only under `benchmark/results/` and must never replace
+root/docs/history artifacts. `--dtypes f32,float64` selects comma-separated dtypes/aliases (canonical
+set + aliases in `benchmark_modes.py`); a multi-dtype scenario (for example src→dst casts) matches when
+any dtype side is requested.
+
 The join is by **normalized op name**: `normalize_op_name` strips the dtype tag, `[annotations]`, and
 identifier-only arg lists — so C# `"np.foo(a)"` and NumPy `"np.foo"` both collapse to `np.foo` and join. Get the
 names to normalize identically or the row shows as "C# not run" / "NumPy only".
 
 ## The matrix + subsystems
 
-- **Op matrix** — 14 comparison suites, each a C# namespace filter in `run_benchmark.py`'s `SUITES` map
+- **Op matrix** — 18 comparison suites, each a C# namespace filter in `run_benchmark.py`'s `SUITES` map
   (`arithmetic, unary, reduction, broadcast, creation, manipulation, slicing, comparison, bitwise, logic,
-  statistics, sorting, linalg, selection`). Swept over 1K/100K/10M × the 15 dtypes.
+  statistics, sorting, linalg, selection, fft, random, ndarray, api`). Universal-tier rule: an
+  operation/dtype with both 1K and 100K must also schedule 10M; the official merge checks NumPy and C#
+  independently. Scalar/1K-only dispatch cases are the only intentional non-throughput exception.
+  Allocation-heavy families map the 10M label to 1M physical elements symmetrically on both sides;
+  1M is not a separate report/dashboard tier.
+- **Backend profiles** — the complete official LinearAlgebra BDN classes run under both the Core-only
+  executable and `NumSharp.Benchmark.CSharp.OpenBLAS`; a merge gate rejects any missing exact-cell OpenBLAS
+  peer. `benchmark/backends/backend_profiles.py` supplements backend-only product/LAPACK routes with
+  the same schema; MissingBackendException and NotSupportedException are availability outcomes. Every
+  profile publishes `1K / 100K / 10M`; operation-specific physical work remains bounded (LAPACK maps
+  those tiers to matrix sides `32 / 96 / 128`). Separate profile JSON files are merged into one
+  effective dataset, with `actual_backend: managed` retained for controls that never dispatch to BLAS.
 - **Five appended subsystems** (own result models, appended not merged): `nditer` (iterator machinery),
   `layout` (op × 8 memory layouts × dtype), `operand` (1-D/scalar/mixed/broadcast), `cast` (astype 15×15 × layout),
-  `fusion` (`np.evaluate`). Each is a `*_bench.{cs,py}` pair + a `*_sheet.py` renderer.
+  and `fusion` (`np.evaluate`). Each is a
+  `*_bench.{cs,py}` pair + a `*_sheet.py` renderer.
 
 ## Playbook — add a benchmark for a new op
 
@@ -65,14 +97,19 @@ The most common task. Full worked example in **`references/add-benchmark.md`**.
 2. **NumPy twin** — append to the matching `run_<suite>_benchmarks(...)` in `numpy_benchmark.py`, setting
    `r.name, r.category, r.suite, r.dtype`. Make `.name` normalize to the C# Description (`"np.foo"` ↔ `"np.foo(a)"`).
 3. **Smoke it** (this is usually the right scope — a full measured run is the post-release CI job):
-   `dotnet build -c Release`; `dotnet run -c Release --no-build -f net10.0 -- --list flat | grep <Class>` to confirm
-   BenchmarkDotNet discovers it; `python numpy_benchmark.py --suite <suite> --quick` to confirm the NumPy rows emit.
-4. **Full numbers** come from `python run_benchmark.py` (or the `benchmark.yml` post-release workflow).
+   `python benchmark/run_benchmark.py --depth pass --suites <suite> --dtypes <dtype>` executes each
+   selected BDN/NumPy cell exactly once and fails on workload errors. Use `--depth light` for a rough ratio.
+4. **Full numbers** come from `python run_benchmark.py --depth measure` (or the `benchmark.yml` post-release workflow).
 
 ## Other tasks → where to go
 
-- **Run the suite (official / subset), interpret the report, snapshots** → `references/run-and-report.md`.
-- **Add or edit a matrix subsystem (nditer/layout/operand/cast/fusion)** → `references/subsystems.md`.
+- **Run the suite (official / subset), interpret the report, the reports/UI surfaces + snapshots** → `references/run-and-report.md`. (The human-facing UI is the DocFX page `docs/website-src/docs/benchmarks-dashboard.md`; its Function Explorer data is generated, while narrative cards are curated. Generated dashboard data is delivered via the orphan **`master-code-data`** branch with a build-time date-priority resolver in `tools/dashboard_data/` — see `references/run-and-report.md`.)
+- **Add or edit a matrix subsystem or backend profile case** → `references/subsystems.md`.
+- **"Is `np.<foo>` benchmarked yet? What's still missing?"** → the generated coverage ledger
+  `benchmark/coverage/generated/summary.md` (+ `coverage.{json,csv}`), refreshed by `scripts/audit_coverage.py`
+  from the `[Benchmark(Description)]` attributes across the op-matrix namespaces plus the reviewed
+  `coverage/overrides.json`. Its "Missing benchmark coverage" table is the to-do list; its route map says which
+  APIs need OpenBLAS. This is a source-level audit, not a timing run.
 - **Everything else (all suites, config internals, troubleshooting, type map)** → `benchmark/CLAUDE.md`.
 
 ## Gotchas
@@ -83,12 +120,18 @@ The most common task. Full worked example in **`references/add-benchmark.md`**.
   trying to run BDN ad-hoc.
 - **What we commit is `benchmark/history/<date>_<sha>/`**, not the gitignored `benchmark/results/<ts>/` scratch.
   Reference `benchmark/history/latest/benchmark-report.md`.
+- **A row shows "C# not run" / "NumPy only"?** The two sides' names didn't normalize to the same join key — the
+  first thing to check for any new/renamed benchmark. `scripts/check_smoke_joins.py` verifies every C#
+  `[Benchmark(Description)]` ↔ NumPy name join in both directions using the exact merge normalizer (from a quick
+  `numpy_benchmark.py --quick --size small` smoke run), so it catches the mismatch without a full measured run.
 - **These are mostly view ops → sub-µs.** flip/rot90/transpose-aliases are O(1) views; their benchmark tracks
-  allocation/dispatch overhead, not throughput. Ops doing real work (trim_zeros, reductions) are where ratios
-  are meaningful.
+  allocation/dispatch overhead, not throughput. `benchmark/scripts/credibility.py` marks every reviewed
+  O(1)-in-N scenario negligible regardless of measured duration, and every dashboard rollup must honor that
+  status. The proof ledger is `benchmark/O1_EXCLUSIONS.md`. Ops doing real work (trim_zeros, reductions) are
+  where ratios are meaningful.
 
 ## References
 
 - `references/add-benchmark.md` — the detailed add-a-benchmark playbook (C# + NumPy twin + join-key rules + smoke).
 - `references/run-and-report.md` — running the official run / subsets, the report + history snapshots, InProcessEmit.
-- `references/subsystems.md` — the five subsystems and how to add one (`*_bench.{cs,py}` + `*_sheet.py`).
+- `references/subsystems.md` — backend profiles plus the five appended subsystems.

.claude/skills/benchmark/references/add-benchmark.md

@@ -96,14 +96,20 @@ A full measured run is expensive and normally the post-release `benchmark.yml` j
 ```bash
 cd benchmark/NumSharp.Benchmark.CSharp
 dotnet build -c Release -v q --nologo
-dotnet run -c Release --no-build -f net10.0 -- --list flat | grep FlipRot     # BDN discovers the 7 methods
+dotnet run -c Release --no-build -f net10.0 -- --list flat | grep FlipRot     # BDN discovers the FlipRot methods
 cd ../NumSharp.Benchmark.Python
 python numpy_benchmark.py --suite manipulation --quick --size medium | grep -iE "flip|trim_zeros"   # NumPy rows emit
 ```
 
 `--list flat` is reflection-only (no toolchain), so it works despite the out-of-process-toolchain limitation.
 A direct-call `dotnet run -c Release` script (mirroring each benchmark body) confirms the C# calls execute.
 
+Two source-level checks confirm the wiring without a measured run: `python benchmark/scripts/check_smoke_joins.py`
+(after the smoke run above) proves the C# `[Benchmark(Description)]` and the NumPy `.name` normalize to the same
+join key in both directions; and — if you benchmarked a *new* API — `python benchmark/scripts/audit_coverage.py`
+refreshes `benchmark/coverage/generated/summary.md`, which should now count your op and drop it from that file's
+"Missing benchmark coverage" table (register a deliberate non-benchmark in `coverage/overrides.json`).
+
 ## 5. Full run (optional)
 
 `python run_benchmark.py --suites manipulation` measures just your suite; `python run_benchmark.py` runs everything
@@ -121,7 +127,7 @@ Only when your ops form a new category with no existing home:
 4. (legacy) optionally add the filter to `run-benchmarks.ps1`'s `ValidateSet` and Program.cs menu.
 
 Then the new suite runs in `python run_benchmark.py --suites <newsuite>` (and in the full run), joined and reported
-exactly like the built-in 14. Keep the C# `[Benchmark(Description)]` labels normalizing onto the NumPy `.name`s.
+exactly like the built-in 18. Keep the C# `[Benchmark(Description)]` labels normalizing onto the NumPy `.name`s.
 
 ## Choosing base class & size (quick reference)