stats API: say where the traffic came from - #279
Merged
Merged
Conversation
`crawlproof stats` could show what a caller SAYS it is — its source, its referrer, the pages it asked for — and every one of those is set by the client. Where the packets entered from is not, and that panel rendered only in the dashboard, so the terminal could not settle the one question that distinguishes an audience from an operator. It matters more than it sounds. Traffic that reads as a broad readership in Sources and resolves to a single city here is one machine, and a referrer panel full of your own hostname will never say so. `countries` and `cities` already existed in PANEL_KEYS and already had their RPCs; they were simply not in STATS_PANELS, so the API never asked for them. Two extra queries per call, which the panel comment now justifies rather than leaving to be rediscovered. Both CLIs share lib/dashboard/stats-text, so rendering is one change. Empty panels print nothing, like every other section. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush Security Scan48 finding(s) HIGH/CRITICAL: 2 | MEDIUM: 31 | LOW: 15
Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
crawlproof statscould show what a caller says it is — its source, its referrer, the pages it asked for. All three are set by the client. Where the packets entered from is not, and that panel rendered only in the dashboard, so the terminal could not settle the one question that separates an audience from an operator.This came out of chasing a suspected scrape on another property: the referrer panel showed traffic that looked scattered, and the geography showed it was essentially one city. Nothing in the CLI could show that.
Change
countriesandcitieswere already inPANEL_KEYSwith their RPCs (tracker_top_countries/tracker_top_citiesand the_recent_variants) — they simply were not inSTATS_PANELS, so the API never asked for them.lib/tracker/apiStats.ts— both panels added toSTATS_PANELSand toStatsAnswer.lib/dashboard/stats-text.ts—CountriesandCitiessections. Both CLIs (in-repo and the published@profullstack/crawlproof) import this renderer, so that is one change rather than two.Cost is two extra queries per call. The panel comment now says why that is worth paying rather than leaving it to be rediscovered — the existing comment justified excluding panels, and this is the exception to it.
Testing
tests/dashboard-package-cli.test.ts— 13 pass, including two new: geo sections render, and empty panels stay silent.tracker-api-stats,tracker-stats-failure,ads-stats-box,tracker-visitor-rollup— 52 pass.tsc --noEmitclean.🤖 Generated with Claude Code