Skip to content

Record where downloads come from, and GitHub's download totals - #79

Merged
yaacovcorcos merged 3 commits into
mainfrom
feature/download-sources-20261117
Nov 17, 2026
Merged

yaacovcorcos merged 3 commits into
mainfrom
feature/download-sources-20261117

Conversation

@yaacovcorcos

@yaacovcorcos yaacovcorcos commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Answers "where do downloads come from?":

  • Source. The first page view of a visit records the referring site's host (never its full address) and the link's utm_source, utm_medium and utm_campaign tags. The browser keeps them for the tab only. The site's download links carry them, plus the page the download was clicked on. A download link on another site is credited to that site's host.
  • Place and network. Already recorded for every website event (analytics_web_sightings). Its referrer column used to hold this website's own host; it now holds the real referring site.
  • Downloads straight from GitHub. The events Worker saves GitHub's running total for each release file once a day (github_release_downloads, migration 0009). Mac .dmg totals are installs only. Windows .exe and Linux .AppImage totals also include updates that fell back to GitHub from the update mirror. Updates the app makes are counted separately by the app (app.update.outcome).
  • Report. bun run analytics:report shows 30-day downloads by source, campaign, place, network and GitHub's totals.

Values that look like an address, a path or an email are dropped before storage.

Documentation and source-version impact

Documentation impact: Updated — README.md (Website analytics: visit source and GitHub totals; repository family line for scient-agent) and src/pages/privacy.astro (installers are served by ScientFactory or GitHub). The owner chose this wording: the existing "usage, device and network information to understand how people find and start using Scient" already covers the visit source, so nothing else was added.

Validation

  • bun run check (format, typecheck, 199 tests, site build, worker builds)
  • Desktop and relevant mobile preview evidence recorded, with limitations stated — no visible UI change. The built browser script was run in Node against browser stand-ins: the landing view sent referrer_host and the tags, later views sent nothing extra, and download links carried the source while other links were untouched. Not yet observed in a real browser or on the Cloudflare preview.
  • Product owner authorized merge/auto-merge, or requested manual preview review completed — the owner approved the privacy wording and authorized merge on 2026-11-18. No manual preview review was done; evidence above is agent-operated.

Live check: https://scientfactory.com/download?utm_source=check 301s to /about/?utm_source=check#downloads, so links to /download keep their tags and referrer.

Quality Review

Light Codex review of b53b1c7: four findings, all confirmed and fixed in the next commit.

  1. Campaign tags with an email or a path reached sessionStorage, the download URL, and (for host/path) D1. Fixed: tags must be short labels with no / or @, checked in the browser before storage and again on the server. Re-ran the built script: utm_campaign=person%40example.com and utm_medium=example.com/private/path are dropped.
  2. page and landing_path accepted //example.com/... and paths with an email. Fixed in cleanPath, which now also covers page views.
  3. A beta-repository GitHub failure discarded stable's totals. Fixed: each repository is saved on its own; the next run retries only the one that failed. Tested.
  4. Direct visits still wrote this website as the sighting's referrer from the request's own Referer. Fixed: only the visit's source (or the other site whose download link was followed) is recorded. Tested.

Integration Readiness Review

Head 4506407. Author review after the Codex fixes: all four findings fixed and tested; the privacy commit changes one sentence and the date. No open findings. Evidence gap: the browser script was checked against stand-ins in Node, not in a real browser; the first real check is a tagged visit on production after deploy. Production impact: download redirect targets unchanged; new event properties only; migration 0009 is additive.

Deployment impact

  • Apply migration 0009 (bun run db:migrate) before deploying the events Worker (bun run events:deploy); Pages code does not need the new table.
  • Optional GITHUB_TOKEN Worker secret raises GitHub's rate limit; without it the job uses public, unauthenticated requests at most once an hour until a day is saved.
  • Redirect targets for downloads are unchanged; download links only gain query parameters.

🤖 Generated with Claude Code

- The first page view of a visit records the referring site's host (never its
  full address) and the link's utm_source, utm_medium and utm_campaign tags.
- The site's download links carry that source and the page they were clicked
  on; a download link on another site is credited to that site's host.
- Web sightings now keep the real referring site instead of this website.
- The events Worker saves GitHub's per-file release totals once a day, so
  downloads made straight from GitHub are counted. Windows and Linux totals
  include updates that fell back to GitHub.
- The analytics report shows 30-day downloads by source, campaign, place,
  network and GitHub's totals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Deploying scientfactory-website with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4506407
Status: ✅  Deploy successful!
Preview URL: https://1e4d2c05.scientfactory-website.pages.dev
Branch Preview URL: https://feature-download-sources-202.scientfactory-website.pages.dev

View logs

yaacovcorcos and others added 2 commits November 18, 2026 00:01
…each repo's totals

- Campaign tags must be short labels with no "/" or "@", checked in the
  browser before anything is stored or sent, and again on the server.
- Page and landing paths must be paths on this site: "//host/..." and paths
  with an email are dropped.
- Web sightings record only the visit's source, never this website's own
  page from the request's Referer.
- A GitHub failure for one repository no longer discards the other's totals;
  the next run retries only the repository that failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yaacovcorcos
yaacovcorcos marked this pull request as ready for review November 17, 2026 22:08
@yaacovcorcos
yaacovcorcos merged commit 1cf708c into main Nov 17, 2026
2 checks passed
@yaacovcorcos
yaacovcorcos deleted the feature/download-sources-20261117 branch November 17, 2026 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant