fix(leads): report a login wall as a login wall, not "no businesses found" - #128
Merged
Merged
Conversation
…ound" Seeding a page that requires a login reported the same thing as seeding an empty directory, which sent users off to debug their search terms when they had simply never been shown the page. Status codes are no help. Instagram answers a search URL with HTTP 200, having quietly redirected to /accounts/login/?next=<the page you asked for>, so a 200 carrying no business links is indistinguishable from a directory with nothing on it. Detection keys off two things instead. A redirect onto a login path that carries a return parameter is the site explicitly refusing the request, and it is visible to a plain fetch, so the common case costs no render. A password field in the DOM covers login screens that only exist once scripts have run. Both are needed: Instagram's redirect shows up in fetch, while its login form does not appear until the page renders. Care went into not misfiring, since a false positive would break a seed that works today. A "Log in" link in a directory's header is not a wall, seeding a login page on purpose is not a wall, an ordinary redirect is not a wall, and a bot challenge is a different failure that still reports as one. The result now carries loginRequired, so the UI can offer to store credentials for that host rather than showing a dead end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vu1nz Security Review0 finding(s) in PR #? No security issues found. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seeding a page that requires a login reported the same thing as seeding an empty directory, sending users off to debug their search terms when they had simply never been shown the page.
Why status codes don't work
Instagram answers
explore/search/keyword/?q=...with HTTP 200, having quietly redirected to/accounts/login/?next=<the page you asked for>. A 200 carrying no business links is indistinguishable from a directory with nothing on it — which is exactly how it surfaced.Two signals instead
Both are needed here: Instagram's redirect shows up in
fetch, while its login form does not appear until the page renders.Not misfiring
A false positive would break a seed that works today, so the negative cases are tested as carefully as the positive one. Verified quiet on:
/browse→/browse/page-1)Also handles cross-host SSO bounces and malformed URLs.
Result shape
discoverFromSeedanddiscoverProspectsnow carryloginRequired/loginRequiredSeeds, so the UI can offer to store credentials for that host instead of showing a dead end. Credential storage itself is a follow-up.Verification note
Instagram began returning 429 to this host after repeated probing, so the live end-to-end path could not be re-confirmed. The tests pin the exact real redirect URL captured before the throttling, so detection is verified against real-world data rather than invented fixtures.
Checks
tsc --noEmitclean🤖 Generated with Claude Code