Skip to content

Shared lexicon (a): one bullet-glyph vocabulary — ⁃ and — are parser bullets the scorer silently drops #915

Description

@s-annam

Part (a) of #653. Not a good first issue, and do not batch it — this one changes scoring behaviour and needs a reviewed, field-by-field rebaseline.

Decisions (2026-09-29)

  • One shared glyph vocabulary, one import-nothing leaf. Parser and scorer both derive their bullet classes from it. No second hand-written copy anywhere.
  • The scorer adopts the parser's glyphs. ⁃ (U+2043) and — (em dash) become scorer bullets. The scorer keeps the glyphs it already recognises that the parser lacks (▶ ► · �) — dropping them would move scores the other way for no reason; the implementer verifies the actual union from the current sources (see the table) and records each glyph's adjudication in the leaf docblock.
  • Leading hyphen - and en-dash – are the ambiguous pair (date-range fragment / negated number, see cleanRewriteLine eats the minus sign off a line-initial negative number (-5% churn → 5% churn) #821). They are expressed as an explicit, documented named subset in the leaf (e.g. AMBIGUOUS_DASH_BULLETS), not a second copy of the list, with the cleanRewriteLine eats the minus sign off a line-initial negative number (-5% churn → 5% churn) #821 reason in the docblock. Maintainer intent: these two stay parser-only — i.e. the scorer must not gain new hyphen/en-dash handling from this issue. NOTE: the scorer's classes today (BULLET_MARKER_RE, apply-overrides.ts LEADING_MARKER_RE) already include - and –. The implementer must not silently drop them from the scorer (that would deflate Specificity on every hyphen-bulleted résumé); if the maintainer's "parser-only" is read literally, stop and ask before removing them. Default if unanswered: the scorer's current -/– handling is unchanged, and the named subset documents that they are the only glyphs the parser and scorer treat by separate rules.
  • Score change and a corpus/score rebaseline are allowed. The implementer must justify every moved baseline field by field (fixture, which glyph, before → after Specificity/bullet count) in the PR description. Bump ATS_SCORE_ALGO_VERSION (currently "1.8", src/lib/score/score.ts) and add the version's entry to its docblock changelog, following the 1.x entries (which glyph set changed, that WHICH bullets are pooled moves, so Specificity/Structure move). The bump also invalidates the Local-first storage foundation: IndexedDB module for resumes and jobs #321 resume-library cache, which is intended.

Problem

The bullet-glyph vocabulary exists as divergent copies, and the divergence is live: glyphs the parser treats as bullets are invisible to the scorer, so those bullets drop out of the Specificity pool.

Verified divergence (line numbers re-checked 2026-09-29; they drift, so anchor on the symbol):

site class
src/lib/heuristics/line-primitives.ts:41 (BULLET_CLASS) [•‣▪●◦⁃*\-–—]
src/lib/heuristics/sections.ts:152 (VISUAL_BULLET_RE) [•‣▪●◦⁃*\-–—]
src/lib/heuristics/regex.ts:383 (LEADING_BULLET_RE) [•‣▪●◦⁃*\-–—]
src/lib/score/score.ts:137 (BULLET_MARKER_RE) [-*•●–▪◦‣▶►·�]
src/lib/score/score.ts:831 (LONE_BULLET_RE) [•●▪◦‣▶►·�]
src/lib/edit/apply-overrides.ts:176 (LEADING_MARKER_RE) [-*•●–▪◦‣▶►·�]
src/lib/score/group-bullets.ts:35 (LEADING_MARKER_RE) same as the scorer's, plus numbered markers
src/lib/heuristics/markdown-emit.ts:66 (LEADING_BULLET_RE) [•▪◦‣·⁃∙●⬤▸▶*\-]

Also near-copies to reconcile or consciously leave (say which in the PR): src/lib/heuristics/extract/summary.ts:18 (SUMMARY_BULLET_RE), src/lib/heuristics/localize/achievements.ts (mirrors group-bullets' marker set), and the comment-only mirror in extract/skills.ts:24. (The original issue listed extract/group-bullets.ts and webllm/localize/achievements.ts; those paths are now score/group-bullets.ts and heuristics/localize/achievements.ts.)

⁃ (U+2043 hyphen bullet) and — (em dash) are parser bullets absent from the scorer's class. A résumé using either has its Experience bullets segmented correctly by the parser and then silently skipped by extractBulletsFromLines (score.ts ~:851), deflating Specificity. Conversely ▶, ►, · and � are scorer-only.

Why this is separate from #653(b)/(c)/(d)

Merging the sets changes scores on existing résumés. That is a deliberate behaviour change requiring a version bump and a reviewed rebaseline, which is why it cannot ride along with the behaviour-preserving consolidations.

Implementation plan

  1. Mint a new import-nothing leaf for the glyph vocabulary. House precedent is extract/title-shape.ts. Not line-primitives.ts — it imports regex.ts, and deriving there re-creates the cycle regex.ts (LEADING_BULLET_RE's comment) exists to avoid, dragging the large regex.ts and its module-eval Intl gazetteer onto the eager entry graph via the score/edit consumers (the feat: primary role targeting, editable headline, and job-search handoff (#598) #605 failure class).
  2. Adjudicate each glyph in the union: parser-bullet / scorer-bullet / both, with the reason written in the leaf docblock. Per the Decisions above: the union is shared, -/– are the documented ambiguous subset.
  3. Derive all sites above from the leaf as named exported sets, preserving any distinction that survives adjudication (the ambiguous-dash subset; any scorer-only glyph) as its own derived constant rather than collapsing it.
  4. Bump ATS_SCORE_ALGO_VERSION and add its changelog entry.
  5. Rebaseline the corpus snapshots (npm run bake-fixtures); review the score deltas fixture by fixture and explain each in the PR description.

Acceptance criteria

  • One import-nothing leaf owns the glyph vocabulary; every site in the table derives from it (no remaining hand-written glyph class), and any near-copy left alone is named with a reason in the PR.
  • The leaf imports nothing, asserted by a test (there is no existing test for title-shape.ts's contract — write one; scripts/check-eager-graph.mjs is the eager-graph counterpart).
  • Per-glyph adjudication, including the -/– named subset and the cleanRewriteLine eats the minus sign off a line-initial negative number (-5% churn → 5% churn) #821 reason, is written in the leaf's docblock, not just in the PR.
  • The scorer treats ⁃ and — as bullets; the scorer does not lose any glyph it recognises today.
  • ATS_SCORE_ALGO_VERSION bumped with a docblock changelog entry.
  • Corpus rebaseline reviewed fixture-by-fixture; every moved baseline justified field by field in the PR description.
  • A regression test covers a ⁃-bulleted and an —-bulleted résumé reaching the Specificity pool.
  • The / entry chunk does not gain regex.ts (build-verified; npm run check:eager passes).

Parent: #653. Roadmap item 5(a) of #646.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    gaalHand this issue to Gaal, the repo's coding agentimprovementEnhancing existing functionalityrefactorCode restructuring without behavior change

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions