Skip to content

Trending scrape records "1 star today" as 0 — the regex only matches the plural "stars" #36

Description

@DamengRandom

Where

src/tools/trending-scrape.tool.ts:32-33

// Stars today
const todayMatch = block.match(/([\d,]+)\s+stars\s+today/)
const todayStars = todayMatch ? parseInt(todayMatch[1].replace(/,/g, ''), 10) : 0

The pattern requires the literal stars. GitHub renders the singular 1 star today when a repo gained exactly one star, so the match fails and line 33 falls back to 0.

Reproduction

This is live production data, not a constructed input — https://github.com/trending/javascript?since=daily is one of exactly two pages this job scrapes (src/agent/index.ts:151 passes ['typescript', 'javascript']).

A throwaway script in the repo root, run with npx tsx, that pulls the page, finds which repo GitHub says gained one star, then runs the real trendingScrapeTool over the same page:

GitHub says these gained exactly 1 star today: [ 'OWASP/threat-dragon' ]
INFO: 🔍 Scraped 14 trending repos from GitHub
  OWASP/threat-dragon: GitHub="1 star today"  parsed todayStars=0
control (plural): addyosmani/agent-skills todayStars=226
digest line: ⭐ 500 (+0 today) · JavaScript

The raw markup on the page, for reference:

</svg>\n        1 star today\n</span>  </div>\n</article>

Expected vs actual

Expected: todayStars === 1 — the same number GitHub displays.

Actual: todayStars === 0, indistinguishable from a repo that gained nothing.

That wrong value is then used three times:

  • src/agent/index.ts:166rankByGrowth sorts on todayStars, so the repo is ranked as if it had no growth at all.
  • src/tools/news-telegram.tool.ts:21 — the digest renders ⭐ 1,554 (+0 today), a number that contradicts the GitHub page it links to.
  • src/storage/own-db.ts:127 — persisted to github_trending.today_stars as 0.

Why this is certain

0 is not a defensible reading of 1 star today; it is the miss-fallback firing on input the parser was meant to handle. Every other repo on the same page parses correctly (the control above), so the parser is right in general and wrong only on the singular form. The trigger is not hypothetical or an unreachable edge — it was present on the JavaScript trending page at the time of this scan, and today's Zig page carried two more instances, so GitHub emits the singular form routinely.

Observed vs reasoned

Observed by execution: the live page containing 1 star today; the real trendingScrapeTool returning todayStars=0 for that repo; the plural control parsing correctly; the rendered digest line reading (+0 today).

Reasoned, not observed: the downstream ranking and DB effects. Those follow by reading rankByGrowth and saveTrendingRepos; I could not run them, since the LLM curator and the database both need credentials this environment does not have.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions