Add a filetype for diffing flame graphs - #181
Draft
ESultanik wants to merge 2 commits into
Draft
Conversation
Graphtage reads the folded stacks format that stackcollapse-perf.pl and its siblings emit: one stack trace per line, written as a ';'-delimited list of function names followed by a space and an integer sample count. A flame graph is modeled as a mapping from stack trace to sample count, so a StackTrace is a KeyValuePairNode whose key is the list of frames and whose value is the count. MultiSetEdit's auto_match_keys pass then pairs the stack traces that two profiles have in common, and only the stack traces unique to one profile reach the bipartite matcher. Files ending in .folded or .collapsed are detected by extension, which requires registering the MIME type because mimetypes does not know it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
Filetypefor diffing flame graphs, replacing #50.Graphtage reads the folded stacks format that
stackcollapse-perf.pland its siblings emit: onestack trace per line, written as a
;-delimited list of function names followed by a space and aninteger sample count. Files ending in
.foldedor.collapsedare detected by extension.Why this is a new branch rather than a rebase of #50
#50 dates from January 2022 and is 293 commits behind
master. Three things ruled out merging it:StackTracean__eq__/__hash__that compare only the function names.FlameGraphwas aMultiSetNode, soMultiSetNode.edits()compared the children asHashableCounters, and two profiles differingonly in sample counts have equal counters. The result is
Match(self, node, 0). The workedexample in Adds diffing capability for flame graphs #50's description was generated before that commit, so it does not show this.
bounds.pyhunk wraps the pre-heapmake_distinctloop that perf: Use heap in make_distinct() for O(n log n) complexity #110 rewrote. TheContainerNode.__eq__it adds duplicates the one atsequences.py:185and compares withzip, so[a] == [a, b]. TheCompoundEditdefaultsduplicate
edits.AbstractEdit. DeletingKeyValuePairNode.__eq__/__hash__would drop thecached hash at
graphtage.py:190. Both progress bars bindDEFAULT_PRINTERat import, which Make --no-status and --quiet suppress the progress bar #135moved behind
get_default_printer()so--quietis honored.SetNode/SetEditrefactor has no caller.builder.build_setstill returns aMultiSetNode.This branch keeps only
flamegraph.py's ideas and touches no core module.Node model
A flame graph is a mapping from stack trace to sample count, so it is built as one:
StackFramesListNode[StringNode];-delimited framesStackTraceKeyValuePairNodekey=StackFrames,value=IntegerNodeFlameGraphDictNodeMultiSetEdit's existingauto_match_keyspass then pairs the stack traces two profiles have incommon and emits a
KeyValuePairEditfor a changed count, so no new edit type is needed and only thestack traces unique to one profile reach the bipartite matcher.
Other fixes relative to #50
to_obj()would have raisedTypeError.MappingNode.to_objuses the key'sto_obj()as a dictkey, and a frame list is not hashable, so
--match-ifand--match-unlesswould have crashed.FlameGraph.to_obj()now returns{"main;work": 100, ...}.ord(), which mis-parses a frame name ending ina digit.
rpartition(" ")also handles frame names containing spaces, such as C++ signatures.get_filetypeis not carried over. It probed every unrecognized file behind abare
except:;__main__.pyalready reports the resultingValueErrorcleanly.Performance
Diffing large profiles is expensive, and the README says so. The cost is not in this module: profiling a
1,000-stack diff puts roughly half the time in
intervaltreechurn insidemake_distinctand the restin Levenshtein tightening in the bipartite matcher. An equivalent 975-key JSON dictionary built from the
same data takes 44 s against 34 s for the flame graph, so this is the existing unordered-matching cost
rather than anything new. Flame graphs just make it easy to reach, since folded files routinely have
thousands of stack traces.
Measured on this branch: 500 stack traces 4.3 s, 1,000 stack traces 34 s.
Tests
test/test_flamegraph.pycovers the diff path, which@filetype_testcannot reach because it onlyround-trips unedited trees: a changed sample count, an added and a removed stack trace, a removed
frame, identical inputs,
to_obj, frame names containing spaces, and the three parse errors.test_flamegraph_formattingintest/test_formatting.pysatisfiestest_formatter_coverage.test/test_main.pycovers extension detection for.foldedand.collapsed.Each new test was confirmed to fail when the behavior it covers is reverted. In particular,
re-adding #50's frames-only
__eq__/__hash__failstest_sample_count_change_is_reportedandnothing else.
🤖 Generated with Claude Code