Skip to content

fix(serve): rank resolve_element ties by closeness, not the alphabet - #70

Merged
jfrench9 merged 1 commit into
mainfrom
bugfix/resolve-element-ranking
Sep 16, 2026
Merged

jfrench9 merged 1 commit into
mainfrom
bugfix/resolve-element-ranking

Conversation

@jfrench9

@jfrench9 jfrench9 commented Sep 16, 2026

Copy link
Copy Markdown
Member

Summary

resolve_element scores matches in tiers (exact, prefix, token, label, qname), and a phrase like "revenue" prefixes hundreds of concepts in one tier. Ties fell to fact count and then the alphabet. A filing's fact counts usually separate them; a taxonomy loaded in 0.17.0 reports nothing, so on US GAAP "revenue" answered RevenueChangeInJudgment and "Revenue Commissioners, Ireland [Member]" first and ranked us-gaap:Revenues past 100. This ranks ties by closeness instead, and scores a label that differs from the phrase only by punctuation or a plural as exact.

Changes

serve/tools.py (surface)

  • Within a score tier: concepts with more facts first (unchanged), then non-abstract before abstract, then numeric before non-numeric (an amount before a text block, a policy or an enumeration), then the shortest concept name, then the qname.
  • A phrase whose squashed form (case, spacing, punctuation removed, one trailing s ignored) equals the concept's name or preferred label scores 90, the exact tier. "operating lease liability" now matches the label "Operating Lease, Liability"; "cash and cash equivalents" matches US GAAP's singular standard label "Cash and Cash Equivalent".

Output Impact

INTERNAL for serialization: holon, TAVI and model output are unchanged. The order of resolve_element results changes wherever a tier ties on fact count. That is always the case on a taxonomy and occasionally on a filing. Scores, match counts and match fields are unchanged. Anything that measures resolve_element rankings, such as the financial analysis benchmark, can see different first matches when run against this version.

Testing

just test-all via the commit hook: ruff, format, basedpyright clean; 545 passed, 2 skipped. The new test (test_resolve_element_ranks_a_taxonomy_with_no_facts_to_lean_on) fails on main (RevenueChangeInJudgment first) and passes here.

Rank of the expected concept, before → after, over 19 queries per source:

Query US GAAP 2025 (taxonomy) NVDA 10-K 0001045810-26-000021
revenue → Revenues 129 → 1 1 → 1
operating lease liability → OperatingLeaseLiability 18 → 1 12 → 1
cash and cash equivalents → CashAndCashEquivalentsAtCarryingValue 16 → 1 4 → 1
share-based compensation → ShareBasedCompensation 206 → 1 34 → 1
inventory → InventoryNet 44 → 1 2 → 1
earnings per share → EarningsPerShareBasic 2 → 1 1 → 1
stockholders equity → StockholdersEquity 16 → 1 1 → 1
research and development → ResearchAndDevelopmentExpense 18 → 1 1 → 1
net income → NetIncomeLoss 5 → 2 (ProfitLoss first) 1 → 1
accounts receivable → AccountsReceivableNetCurrent 26 → 5 (AccountsReceivableNet first) 1 → 1

The other queries (revenues, cash, assets, goodwill, long-term debt, income tax expense, depreciation, operating income) were 1 before and stay 1 on both sources, except that NVDA does not report us-gaap:Cash. A qname query (us-gaap:Assets) ranks 1 on every source. After the change, every query on AAPL 10-Q 0000320193-26-000020 ranks the expected concept 1 wherever the filing reports it, and all six GASB 2026 exposure-taxonomy queries rank 1 (pension liability was 14 before).

A phrase prefixes hundreds of concepts in one score tier, and ties fell to
fact count and then the alphabet. A filing's fact counts usually separate
them; a taxonomy reports nothing, so US GAAP answered "revenue" with
RevenueChangeInJudgment and ranked Revenues past 100.

- Within a tier: reported concepts first, then an amount before a heading,
  a text item or an enumeration, then the shortest name, then the alphabet.
- A phrase that is the name or label but for spacing, punctuation or a
  plural scores as exact: "operating lease liability" is "Operating Lease,
  Liability", "cash and cash equivalents" is "Cash and Cash Equivalent".
@jfrench9
jfrench9 merged commit fd9164e into main Sep 16, 2026
4 checks passed
@jfrench9
jfrench9 deleted the bugfix/resolve-element-ranking branch September 16, 2026 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant