Commit 51efda7
authored
perf(enrich): --memprofile + --max-buffer-pool + --copy-threads flags (Phase C) (#147)
Phase C of the OOM fix plan. Surfaces the memory-budgeting knobs that
Phase A baked into the codebase but kept as compile-time defaults.
Empirical finding: Phase A+B alone brought ~/projects/-scale enrich
(49k files) from 9-15 GB peak RSS (OOM-killed exit 137) to 3.12 GB —
well under the 4 GiB acceptance bar. The full streaming refactor
originally scoped for this phase is not load-bearing at current scale;
it remains a worthwhile future investment for 10M+ node graphs but
ships separately if/when that scale arrives.
Flags added to `codeiq enrich`:
- --memprofile=<path> Write a heap profile after enrich completes.
For OOM debugging — pair with /usr/bin/time -v.
- --max-buffer-pool=N Cap Kuzu BufferPoolSize in bytes (default 2 GiB).
For hosts where 2 GiB is still too much.
- --copy-threads=N Cap Kuzu COPY FROM parallelism (default
min(4, GOMAXPROCS)).
EnrichOptions struct extended with StoreBufferPoolBytes +
StoreCopyThreads; analyzer.Enrich now routes through
graph.OpenWithOptions with those values.
Plan: docs/superpowers/plans/2026-05-13-enrich-oom-fix.md Phase C.
Verification:
- go test ./... -count=1: 877 pass.
- /tmp/codeiq-c enrich ~/projects/polyglot-bench/airflow recorded
1.27 GB peak RSS via /usr/bin/time -v (down from pre-Phase-A
3.8 GB observed by the research pprof agent).
- ~/projects/ enrich peak RSS: 3.12 GB (below 4 GiB acceptance bar).1 parent 548a5ec commit 51efda7
2 files changed
Lines changed: 45 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
26 | 32 | | |
27 | 33 | | |
28 | 34 | | |
| |||
109 | 115 | | |
110 | 116 | | |
111 | 117 | | |
112 | | - | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
113 | 122 | | |
114 | 123 | | |
115 | 124 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
| 7 | + | |
| 8 | + | |
6 | 9 | | |
7 | 10 | | |
8 | 11 | | |
| |||
11 | 14 | | |
12 | 15 | | |
13 | 16 | | |
14 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
15 | 23 | | |
16 | 24 | | |
17 | 25 | | |
| |||
48 | 56 | | |
49 | 57 | | |
50 | 58 | | |
51 | | - | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
52 | 67 | | |
53 | 68 | | |
54 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
55 | 82 | | |
56 | 83 | | |
57 | 84 | | |
| |||
60 | 87 | | |
61 | 88 | | |
62 | 89 | | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
63 | 96 | | |
64 | 97 | | |
65 | 98 | | |
0 commit comments