Skip to content

Fix stats.countby --by-name sort with mixed value types (SYN-9957) - #5017

Merged
invisig0th merged 2 commits into
masterfrom
SYN-9957
Sep 21, 2026
Merged

invisig0th merged 2 commits into
masterfrom
SYN-9957

Conversation

@invisig0th

@invisig0th invisig0th commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

SYN-9957

stats.countby --by-name leaked a raw python TypeError whenever a tally contained a mix of numeric and non-numeric names. Reported by a Sentinel 1 user via epiphyte.

cli> storm inet:ipv4 stats.countby --by-name :asn
ERROR: TypeError: '<' not supported between instances of 'str' and 'int'

The --by-name sort key returned int() for names which parsed as integers and the original str for names which did not, so a tally containing both handed sorted() a non-orderable mix of types. The common trigger is nodes which do not have the tallied property set, since those tally under the name None.

Changes

Numeric names now sort numerically and non-numeric names sort lexicographically after them, via a type-homogeneous sort key.

s_common.hugenum() is used in place of int(), so fractional names sort numerically too (1, 1.5, 2, 10) with exact decimal comparison and none of the float() rounding fuzz. Two cases are handled explicitly:

  • hugenum('0xzz') raises ValueError from its hex branch rather than a DecimalException
  • hugenum('nan') succeeds, but Decimal('NaN') < Decimal(1) raises InvalidOperation — the same leak class being fixed — so non-finite values are treated as non-numeric

The coerce(indx) closure factory was only ever called as coerce(0), so the unused indx generality is dropped.

Before / After

cli> storm inet:ipv4 stats.countby --by-name :asn      # after
None | 1 | #########################
   4 | 1 | #########################
   1 | 2 | ##################################################
   0 | 1 | #########################

Tests

The regression assertions are appended to the existing test_stormlib_stats_countby so they reuse the Cortex it already stands up rather than booting a second one. The new inet:ipv4 nodes are tagged and lifted by tag, so the existing 15-node fixture and its 18 chart expectations are unaffected.

Coverage:

  • the reported unset-property tally, in both directions
  • a mixed tally covering finite decimals, fractions, hex names, the ValueError arm, the non-finite arm, and plain strings

Passes plain and under SYNDEV_NEXUS_REPLAY=1. 100% patch coverage — synapse/lib/stormlib/stats.py reports 99% with the only miss on StatTally.value(), which is pre-existing and outside this diff.

No docs change: storm_ref_cmd.rstorm renders stats.countby --help dynamically and no option help text changed.

Note on 3.x

synsrc/synapse/lib/stormlib/stats.py in synapse-enterprise carries a byte-identical copy of the buggy if byname: block, and the failure reproduces there as inet:ip | stats.countby :asn --by-name. That needs a separate PR against main; the fix transplants verbatim, but the charts differ because v3 renders the percent column unconditionally instead of behind --percent.

https://claude.ai/code/session_013pyj3S8NddfycqDWj9uBUS

The --by-name sort key returned int() for names which parsed as integers
and the original str for names which did not, so a tally containing both
handed sorted() a non-orderable mix of types and leaked a TypeError. This
happened most often when some nodes did not have the tallied property set,
which tallies under the name None.

Sort numeric names numerically using s_common.hugenum() and non-numeric
names lexicographically after them. Using a hugenum rather than int() also
sorts fractional names numerically. Non-finite values such as nan are
treated as non-numeric since they cannot be ordered.

Claude-Session: https://claude.ai/code/session_013pyj3S8NddfycqDWj9uBUS
@codecov

codecov Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.83%. Comparing base (7736fb2) to head (fba0a5d).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5017      +/-   ##
==========================================
- Coverage   97.89%   97.83%   -0.06%     
==========================================
  Files         308      308              
  Lines       65689    65690       +1     
==========================================
- Hits        64305    64270      -35     
- Misses       1384     1420      +36     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@invisig0th
invisig0th marked this pull request as ready for review September 21, 2026 16:00
The new assertions created a second getTestCore() rather than reusing the
one already stood up by test_stormlib_stats_countby. Append them to the
existing context instead to avoid the extra Cortex boot. The added
inet:ipv4 nodes are tagged and lifted by tag so the existing chart
expectations are unaffected.

Claude-Session: https://claude.ai/code/session_013pyj3S8NddfycqDWj9uBUS
Comment thread synapse/lib/stormlib/stats.py
@invisig0th
invisig0th merged commit 40f6f41 into master Sep 21, 2026
6 checks passed
@invisig0th
invisig0th deleted the SYN-9957 branch September 21, 2026 16:42
@vEpiphyte vEpiphyte added this to the v2.252.0 milestone Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants