Skip to content

Add a filetype for diffing flame graphs - #181

Draft
ESultanik wants to merge 2 commits into
masterfrom
50-flamegraph
Draft

Add a filetype for diffing flame graphs#181
ESultanik wants to merge 2 commits into
masterfrom
50-flamegraph

Conversation

@ESultanik

Copy link
Copy Markdown
Collaborator

Adds a Filetype for diffing flame graphs, replacing #50.

Graphtage reads the folded stacks format that
stackcollapse-perf.pl and its siblings emit: one
stack trace per line, written as a ;-delimited list of function names followed by a space and an
integer sample count. Files ending in .folded or .collapsed are detected by extension.

$ graphtage before.folded after.folded
main;work 100
main;work;parse 40 -> 55
main;work;~~emit~~++flush++ 30

Why this is a new branch rather than a rebase of #50

#50 dates from January 2022 and is 293 commits behind master. Three things ruled out merging it:

  1. It produces an empty diff for the case it exists to serve. Commit b84f138 gave StackTrace an
    __eq__/__hash__ that compare only the function names. FlameGraph was a MultiSetNode, so
    MultiSetNode.edits() compared the children as HashableCounters, and two profiles differing
    only in sample counts have equal counters. The result is Match(self, node, 0). The worked
    example in Adds diffing capability for flame graphs #50's description was generated before that commit, so it does not show this.
  2. Its core-library changes are superseded or regressive. The bounds.py hunk wraps the pre-heap
    make_distinct loop that perf: Use heap in make_distinct() for O(n log n) complexity #110 rewrote. The ContainerNode.__eq__ it adds duplicates the one at
    sequences.py:185 and compares with zip, so [a] == [a, b]. The CompoundEdit defaults
    duplicate edits.AbstractEdit. Deleting KeyValuePairNode.__eq__/__hash__ would drop the
    cached hash at graphtage.py:190. Both progress bars bind DEFAULT_PRINTER at import, which Make --no-status and --quiet suppress the progress bar #135
    moved behind get_default_printer() so --quiet is honored.
  3. The SetNode/SetEdit refactor has no caller. builder.build_set still returns a
    MultiSetNode.

This branch keeps only flamegraph.py's ideas and touches no core module.

Node model

A flame graph is a mapping from stack trace to sample count, so it is built as one:

Class Base Role
StackFrames ListNode[StringNode] the ;-delimited frames
StackTrace KeyValuePairNode key=StackFrames, value=IntegerNode
FlameGraph DictNode the whole file

MultiSetEdit's existing auto_match_keys pass then pairs the stack traces two profiles have in
common and emits a KeyValuePairEdit for a changed count, so no new edit type is needed and only the
stack traces unique to one profile reach the bipartite matcher.

Other fixes relative to #50

  • to_obj() would have raised TypeError. MappingNode.to_obj uses the key's to_obj() as a dict
    key, and a frame list is not hashable, so --match-if and --match-unless would have crashed.
    FlameGraph.to_obj() now returns {"main;work": 100, ...}.
  • Adds diffing capability for flame graphs #50 walked characters off the end of the line with ord(), which mis-parses a frame name ending in
    a digit. rpartition(" ") also handles frame names containing spaces, such as C++ signatures.
  • Content sniffing in get_filetype is not carried over. It probed every unrecognized file behind a
    bare except:; __main__.py already reports the resulting ValueError cleanly.

Performance

Diffing large profiles is expensive, and the README says so. The cost is not in this module: profiling a
1,000-stack diff puts roughly half the time in intervaltree churn inside make_distinct and the rest
in Levenshtein tightening in the bipartite matcher. An equivalent 975-key JSON dictionary built from the
same data takes 44 s against 34 s for the flame graph, so this is the existing unordered-matching cost
rather than anything new. Flame graphs just make it easy to reach, since folded files routinely have
thousands of stack traces.

Measured on this branch: 500 stack traces 4.3 s, 1,000 stack traces 34 s.

Tests

  • test/test_flamegraph.py covers the diff path, which @filetype_test cannot reach because it only
    round-trips unedited trees: a changed sample count, an added and a removed stack trace, a removed
    frame, identical inputs, to_obj, frame names containing spaces, and the three parse errors.
  • test_flamegraph_formatting in test/test_formatting.py satisfies test_formatter_coverage.
  • test/test_main.py covers extension detection for .folded and .collapsed.

Each new test was confirmed to fail when the behavior it covers is reverted. In particular,
re-adding #50's frames-only __eq__/__hash__ fails test_sample_count_change_is_reported and
nothing else.

🤖 Generated with Claude Code

ESultanik and others added 2 commits September 9, 2026 11:19
Graphtage reads the folded stacks format that stackcollapse-perf.pl and
its siblings emit: one stack trace per line, written as a ';'-delimited
list of function names followed by a space and an integer sample count.

A flame graph is modeled as a mapping from stack trace to sample count,
so a StackTrace is a KeyValuePairNode whose key is the list of frames and
whose value is the count. MultiSetEdit's auto_match_keys pass then pairs
the stack traces that two profiles have in common, and only the stack
traces unique to one profile reach the bipartite matcher.

Files ending in .folded or .collapsed are detected by extension, which
requires registering the MIME type because mimetypes does not know it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant